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INTRODUCTION 


This book represents the outcome, after more than a decade, 
of a program of work undertaken jointly with Dr. Arturo Rosen- 
blueth, then of the Harvard Medical School, and now of the 
Instituto Nacional de Cardiologia of Mexico. I those days, 


Dr. Rosenblueth, who was the colleague and collaborator of 


the late Dr. Walter B. Cannon, conducted a monthly series of 
The participants were 


discussion meetings on scientific method. 
al the Harvard Medical School, and we 


out a round table in Vanderbilt Hall. 


The conversation was lively and unrestrained. It was nota place 
Where it was either encouraged or made possible for anyone to 
Stand on his dignity. After the meal, somebody — either one 
of our group or an invited guest — would read a paper on some 
Scientific topic, generally one in which questions of methodology 
Were the first consideration, or at least a leading consideration. 
The speaker had to run the gauntlet of an acute criticism, good- 
natured but unsparing. It was a perfect catharsis for half-baked 
ideas, insufficient self-criticism, exaggerated self-confidence, and 
Pomposity. Those who could not stand the gaff did not return, 
but among the former habitués of these meetings there is more 


than one of us who feels thal they were an important and 
Permanent contribution to our scientific unfolding. 
s or medical scientists. 


Not all the participants were physician , 
One of us, a very steady member, and a great help to our dis- 
Cussions, was Dr. Manuel Sandoval Vallarta, a Mexican like 
Dr. Rosenblueth and a Professor of Physics at the Massachusetts 
Institute of Technology, who had been among my Very first 
Students when I came to the Institute after World War No. I. 
Dr. Vallarta used to bring some of his M. I. T. colleagues along 
lo these discussion meetings, and it was as one of these that I 
first met Dr. Rosenblueth. ! had been interested in scientific 


mostly young scientists 
Would gather for dinner ab 
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method for a long time, and had in fact been a participant in 
Josiah Royce’s Harvard seminar on the subject in 1911-1913. 
Moreover, it was felt that it was essential to have someone 
present who could examine mathematical questions critically. 
I thus became an active member of the group until Dr. Rosen- 
blueth’s call to Mexico in 1944 and the general confusion of the 
war ended the series of meetings. 

For many years Dr. Rosenblueth and I had shared the convic- 
tion that the most fruitful areas for the growth of the sciences 
were those which had been neglected as 
tween the various established fields. 
perhaps beet no man who has had a 
intellectual activity of his day. 


been increasingly the task of spe 
tendency 


a no-man’s land be- 
Since Leibniz there has 
full command of all the 
Since that time, science has 
cialists, in fields which show a 
to grow progressively narrower. A century ago there 
may have been no Leibniz, but there was a Gauss, a Faraday, 
and a Darwin. Today there are few scholars who can call 
themselves mathematicians or phy 


sicists or biologists without 
restriction 


logist or an acoustician or a 
coleopterist. He will be filled with the jargon of his field, and 


will know all its literature, and all its ramifications, but, more 
frequently than not, he will regard the next subject as something 
belonging to his colleague three doors down the corridor, and 


will consider any interest in it on his Own part as an unwarran- 
table breach of privacy, 


These specialized fields are 


A man may be a topo 


: i » the Mexicans, and the Russians — an 
inextricable tangle of exploration, nomenclature, and Jaws. 
There are fields of scientific work 


> 4S we shall see in the body of 
ored from the different sides of 
Ctrical engineering, and neuro- 


—— ec 
te 
= 
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opportunities to the qualified investigator. They are at the 
same time the most refractory to the accepted techniques of 
mass attack and the division of labor. If the difficulty of a 
physiological’ problem is mathematical in essence, ten physio- 
logists ignorant of mathematics will get precisely as far as one 
physiologist ignorant of mathematics, and no further. If a phy- 
siologist, who knows no mathematics, works together with a ma- 
thematician who knows no physiology, the one will be unable 
to state his problem in terms that the other can manipulate, and 
the second will be unable to put the answers in any form that 
the first can understand. Dr. Rosenblueth has always insisted 
that a proper exploration of these blank spaces on the map of 
science could only be made bya team of scientists, each a specia- 
list in his own field, but each possessing a thoroughly sound and 
trained acquaintance with the fields of his neighbors; all in the 
habit of working together, of knowing one another's intellectual 
customs, and of recognizing the significance ofa colleague s nEW 
Suggestion before it has taken ona full formal expression. The 
mathematician need not have the skill to conduct a physiological 
experiment, but he must have the skill to understand one, to 
criticize one, and to suggest one. The physiologist need not be 
able to prove a certain mathematical theorem, but hemust be 
able to grasp its physiological significance and lo tell the mathe- 
matician for what he should look. We had dreamed for years 
of an institution of independent scientists, working together in 
One of these backwoods of science, not as subordinates of some 
Sreat executive officer, but joined by the desire, indeed by the 
spiritual necessity, to understand the region as a whole, and to 
lend one another the strength of that understanding. 

We had agreed on these matters long before we had chore 
the field of our joint investigations and our respective parts in 
them. The deciding factor in this new step was the war. I had 
known for a considerable time that if a national emereeney 
Should come, my function in it would be determined largely 
by two things : my close contact with the program of compu- 
ting machines developed by Dr. Vannevar Bush, and my own 
Joint work with Dr. Yuk Wing Lee on the design of electric net- 
Works. In fact, both proved important. In the summer of 
1940,1 turned a large part of my attention to the development of 
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computing machines for the solution of partial differential equa- 
tions. I had long been interested in these, and had convinced 
myself that their chief problem, as contrasted with the ordinary 
differential equations so well treated by Dr. Bush on his differen- 
tial analyzer, was that of the representation of functions of more 
than one variable. 1 had also become convinced that the pro- 
cess of scanning, as employed in television, gave the answer 
lo that question, and in fact that television was destined to 
be more useful to engineering by the introduction of such new 
techniques than as an independent industry. 

Tt was clear that any scanning process must vastly increase 
the numberof data dealt with as compared with the number 
of data in a problem of ordinary differential equations. To ac- 
complish reasonable results in areasonable time, it thus became 
necessary to push the speed of the elementary processes to 
the maximum, and to avoid interrupling the stream of these 
processes by steps of an essentially slower nature. It also 
became necessary to perform the individual processes with so 
high a degree of accuracy that the enormous repetition of the ele- 
mentary processes should not bring about’a cnmulativ 
SO great as to swamp all accuracy. 
ments were suggested : 


e error 
Thus the following require- 


(1) That the central adding 
computing machine should be 
ding machine, rather th 


and multiplying apparatus of the 
numerical, as in an ordinary ad- 
anona basis of measurement, as in the 
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(5) That the machine contain an apparalus for the storage of 
data which should record them quickly, hold them firmly until 
erasure, read them quickly, erase them quickly, and then be 
immediately available for the storage of new material. 


, These recommendations, together with tentative suggestions 
for the means of realizing them, were sent in to Dr. Vannevar 
Bush for their possible useinawar. Atthat stage of the prepara- 
tions for war, they did not seem to have sufficiently high priority 
to make immediate work on them worth while. Nevertheless, 
they all represent ideas which have been incorporated into ihe 
modern ultra-rapid computing machine. These siotions were 
all very much in the spirit of the thought of the time. and] do 
not for a moment wish to claim anything like the sole respon- 
sibility for their introduction. Nevertheless, they have proved 
useful, and it is my hope that my memorandum had some effect 
ong engineers. Atany rate, as we shall 
see in the body of the book, they are all ideas which are of in- 
terest in connection with the study of the nervous system. 

This work was thus laid on the table, and although it has 
not proved to be fruitless, it led to no immediate project by 
Dr. Rosenblueth and myself. Our actual collaboration resul- 
ted from another project which was likewise undertaken for the 
purposes of the last war- At the beginning of the war, the Ger- 
man prestige in aviation and the defensive position of England 
turned the atlention of many scientists to the improvement 
of anti-aircraft artillery. Even before the war it had become 
clear that the speed of the airplane had rendered obsolete all 
classical methods of the direction of fire, and that it was neces- 
sary to build into the control apparatus all the computations 
necessary. These were rendered much more difficult by the fact 
that, unlike all previously encountered targets, an airplane has 

a velocity which is a very appreciable part of the velocity of 
the missile used to bring it down. Accordingly, it is exceeding- 
ly important to shoot the missile, not at the target, but in such 
a way that missile and target may come together in space at 
some time in the future. We must hence find some method of 


predicting the future position of the plane. 
The simplest method is to extrapolate the present course of the 


in popularizing them am 
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plane along a straight line. This has much to recommend af 
The more a plane doubles and curves in flight, the less 1S its 
effective velocity, the less time it has to accomplish a mission; 
and the longer it remains in a dangerous region. Other things 
being equal, a plane will fly as str, 
However, by the time the first shell has burst, other things are 
not equal, and the pilot will probably zig-zag, stunt, or in some 
other way take evasive action. 

If this action were completely at the disposal of the pilot, and 
the pilot were to make the sort of intelligent use of his chances 
that we anticipate in a good poker player, for example, he has 
so much Opportunity to modify his expected position before the 
arrival of a shell that we should not reckon the chances of hilling 
him to be very good, except perhaps in the case of a very waste- 
ful barrage fire. On the other hand, the pilot does not have a 

free chance to maneuver at his will. For one thing, 
dingly high speed, and any (ce 
€ will produce an acceleration 
ous, and may disintegrate the 
control the plane by moving his 
egimen of flow that is established 
lop. Even when it is fully deve- 
cceleration ‘of the plane, and this 
be converted, first into change of 
ge of position, before it is finally 


F ‘ble. 
aight a course as possibl 


me to deve 
pat merely changes the a 
change of acceleration must 
velocity, and then 
effective. 


ontro] apparatus 
To predict the future 
lion on its past. The 
d by any constructible 
which bear it a certain 
Y apparatus which wé& 
1 Caldwell of the Mass- 
that these Operators seemed 
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worth trying, and he immediately suggested that we try them 
out on Dr. Bush's differential analyzer, using this as a ready- 
made model of the desired fire-control apparatus. We did so, 
with results ‘which will be discussed in the body of this book. 
At any rate, I found myself engaged in a war project, in which 
Mr. Julian H. Bigelow and myself were partners in the invyesti- 
gation of the theory of prediction and of the construction of 
apparatus to embody these theories. 

It will be seen that for the second time ] had become engaged 
in the study of a mechanico-electrical system which was designed 
to usurp a specifically human function — in the first case, the 
execution of a complicated pattern of computation ; and in the 
second, the forecasting of the future. In this second case, we 
should not avoid the discussion of the performance of certain 
human functions. In some fire-control apparatus, it is true, the 
original impulse to point comes in directly by radar, but in 
the more usual case, there is a human gun-pointer or a gun- 
trainer or both coupled into the fire-control system, and acting as 
an essential partofit. Itis essential to know ihercharactenis 
in order to incorporate them mathematically ante the machines 
they control. Moreover, their target, the plane, 1s also humanly 
controlled, and itis desirable to know its performance character- 
istics. 

Mr. Bigelow and ] came to the co 
important factor in voluntary activity is W 


term feed-back I shall discuss this in considerable detail in the 
It is enough to say here that when we 


a given pattern, the difference between 
fe) . . 
performed molion is used as a new 


input to cause the part regulated to move in Sn ane : 
bring its motion closer to that given by the pap cee hie 
One form of steering engine ofa ship carries he © g 


i i -evulates the valves of 
Wheel rom the tiller which so regu 
to an offset fron move the tiller in such a way as to turn 


s so as to bring the other end 


nelusion that an extremely 
hat the control engineers 


appropriate chapters. 
desire a motion to follow 
this pattern and the actually 


the steering engine as to 


these valves tiller turn : 
off. Thus thet : : poses 

Of the valve-reoulating offset amidships, and in that way regis 
the wheel as the angular position of 


the angular position of 
the tiller. Clearly, any 
hampers the motion of the t 


friction oF other delaying force which 
e admission of 


iller will increase th 


14 CYBERNETICS 
steam to the valves on one side, and will decrease it pay ae 
in such a way as to increase the torque tending to bring ee 
to the desired position. Thus the feed back system wae Weel, 
the performance of the steering engine relatively indep 
ad. ; 

ee other hand, under certain conditions of delay a 
feed-back that is too brusque will make the rudder ee a 
and will be followed by a feed-back in the other eee 
which makes the rudder overshoot still more, until the eee te 
mechanism goes into a wild oscillation or hunting, are oe 
down completely. In a book such as that by MeGetl ded 
find a very precise discussion of feed-back, the poet sage ne 
which it is advantageous, and the conditions under bane be: 
breaks down. Itis a phenomenon which we understand very 
thoroughly from a quantitative point of view. +, fe 

Now, suppose that Ipick upa lead-pencil. To do this : is 
to move certain muscles. However, for all of us but a . 
expert anatomists, we do not know what these muscles ton 
and even among the anatomists, there are few if any who ¢ - 
perform the act By a conscious willing in succession of the co - 
traction of each muscle concerned. On the contrary, what x 
will is to pick the pencil up. Once we have determined on ne 
our motion proceeds in such a way that we may say rough _ 
that the amount by which the pencil is not yet picked up} 


Th: Se in full 
decreased at each Stage. This part of the action is not in ft 
consciousness. 


To perform an action in such 
report to the nervous system, con 


amount by which we have failed 
instant. 


ea 
a manner, there must b a 
‘ : 1 
scious or unconscious, of ni 
: $ a 
to pick the pencil up at e@ 


ay be 
€ on the pencil, this report may P 
bu 


If we have our ey 


visual, at least in part, but it is more generally kinaesthetic, 


or to use a term now in vogue, Proprioceptive. If the proprio- 
ceptive sensations are wanti 


ng, and we do not replace them by 
a visual or other Substitute, we are unable to perform the act 
of picking up the pencil, and find ourselves in a state of what 
is known as alaxia, 


An ataxia of this ty 
form of syphilis of the 


: Weis of 6 
pe is familiar in th 

ry 
“snital’ nervous system known as (abe 


1. McColl, L., Servomechanisms. Van Nostrand, 1945. 
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dorsalis,-where the kinaesthetic sense conveyed by the spinal 
nerves is more or less destroyed. 

However, an excessive feed-back is likely to be as serious a 
handicap to ofganized activity as a defective feed-back. In view 
of this possibility, Mr. Bigelow and myself approached Dr. 
Rosenblueth with a very specific question. Is there any 
pathological condition in which the patient, in trying to per- 
form some voluntary act like picking up a pencil, overshoots 
the mark, and goes into an uncontrollable oscillation? Dr. 
Rosenblueth immediately answered us that there is such a well- 
known condition, that it is called purpose tremor, and that it is 
often associated with injury to the cerebellum. 

We thus found a most significant confirmation of our hypo- 
thesis concerning the nature of at least some voluntary acti- 
vity. It will be noted that our point of view considerably trans- 
cended that current among neurophysiologists. The central 
nervous system no longer appears as a self-contained organ, 
receiving inputs from the senses and discharging into the mus- 
cles. On the contrary, some ofits most characteristic activities 
are explicable only as circular processes, emerging from the 
nervous system into the muscles, and re-entering the nervous 
system through the sense organs, whether they be propriocep- 
tors or organs of the special senses. This seemed to us to mark 
a new step in the study of that part of neurophysiology which 
concerns not solely the elementary processes of nerves and 
synapses but the performance of the nervous system as an inte- 
grated whole. 2 

The three of us felt that this new point of view merited a 
Paper, which we wrote up and published 1, Dr. Rosenblueth 
and I foresaw that this paper could only be a statement ot pro- 
gram for a large body of experimental work, and we decided 
thal if we could ever bring our plan for an interscientific insti- 
tute to fruition, this topic would furnish an almost ideal center 
for our activity. 

On the communication engineerin 
become clear to Mr. Bigelow and my: 
control engineering and of communica 
«Behaviour, Purpose & Teleology », Philo- 


g plane, it had already 
self that the problems of 
tion engineering were 


1. Rosenblueth, Wiener & Bigelow, 
sophy of Science, vol. 10, pp- 18-24 (1943). 
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inseparable, and that they centered not around the technique 
of electrical engineering but around the much more fandamen= 
tal notion of the message, whether this should be transmitted 
by electrical, mechanical, or nervous means. The mieHsaye a8 
a discrete or continuous sequence of measurable events distri- 
buted in time — precisely what is called a time-series by the 
Statisticians. The prediction of the future of a message is doné 
by some sort of Operator on its past, whether this operator 1s 
realized by a scheme of mathematical computalion, or by a 
mechanical or electrical apparatus. : 
that the ideal prediction mechanisms which we had at first 
contemplated were beset by two types of error, of a roughly 
antagonistic nature. While the prediction apparatus which we 
at first designed could be made to 
smooth curve to any desired 
refinement of behavior 
increasing Sensilivity, 


In this connection, we found 


anticipate an extremely 
degree of approximation, this 
was always attained at the cost of an 
The better the apparatus was for smooth 
Waves, the more it would be set into oscillation by small depar- 
tures from smoothness, and the longer it would be before such 


oscillations would die out. Thus the good prediction ofa smooth 
Wave seemed to require am 


than the best Possible pre 
choice of the particular appar. 
Was dependent on the stat 
be predicted. This intera 
have somethingin common wj 
measure of position an 
Heisenberg quantum mechanics, as q 
Principle of Uncertainty, 

Once we had clearly gr 


asped that the solution of the problem 
of optimum Prediction w 


as only to be obtained by an appeal to 
the statistics of the time series to be Predicted, it was not difficult 
to make what had originally Seemed to be a difficulty in the the- 
ory of prediction into what was actually an efficient tool for sol- 


ving the problem of prediction, Assuming the statistics of a time 
series, it became Possible to derive 
mean squ 


INTRODUCTION 17 


tor which should reduce into a minimum a specific positive quan- 
tity dependent on this operator. Minimizalion problems of this 
type belong to a recognized branch of mathematics, the calcu- 
lus of variations, and this branch has a recognized technique 

With the aid of this technique, we were able to obtain an expli- 
cit best solution of the problem of predicting the future of a time 
series, given its statistical nature ; and even further, to achieve 
a physical realization of this solution by a constructible appa- 
ratus. : 

Once we had done this, at least one problem of engineering 
design took on a completely new aspect. In general, enginee- 
ring design has been held to he an art ratherthan a science. By 
reducing a problem of this sort to a minimization principle, we 
had established the subjecton a far more scienlific basis. It oc- 
curred to us that this was not an isolated case, but that there was 
a whole region of engineering work in which similar design pro- 
blems could be solved by the methods of the calculus of varia- 
tions, 

We attacked and solved other similar problems by the same 
methods. Among these was the problem of the design of wave 
filters. We often find a message contaminated by extraneous 
disturbances which we call background noise. We then face the 
Problem of restoring the original message, OF the message under 
4 given lead, or the message modified by a given lag, by Ee ope- 
rator applied to the corrupted message. The spon design ee 
this operator and of the apparatus by which it is realized depends 
On the statistical nature of the message and the noise, singly and 


jointly. We thus have replaced in the design of wave filters 
of an empirical and rather hapha- 


a thorough scientific justification. 

In doing this, we have made of communication gg 
design a statistical science, a branch ofstatistical mechanics. he 
Notion of stalistical mechanics has indeed been a on 
every branch of science for more than @ century. We au see 
that this dominance of statistical mechanics mu sete p — 
has a very vital significance for the interpretation of t . : ne 
of time. ‘In the snge of communication engineennes howev : 
the signiticance of the statistical elementis immediately git : 
The transmission of information is impossible save as a rans - 


Processes which were formerly 
zard nature by processes with 
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mission of alternatives. If only one contingency is to be trans- 
mitted, then it may be sent most efficiently and with the least 
trouble by sending no message at all. The telegraph and the 
telephone can perform their function only if the messages they 
transmit are continually varied in a manner nol completely 
determined by their past, and can only be designed effectively if 
the variation of these messages confo 
tical regularity. 
To cover this as 
developa statistic 


tms to some sort of statis- 


pect of communication engineering, we had to 
al theory of the amount of information, in which 
the unit amount of information was thattransmitted asa single de- 
cision between equally probable alternatives. Thisidea occurred 
at about the same time to several writers, among them the statisti- 
cian R. A. Fisher, Dr. Shannon of the Bell Telephone Labora- 
tories, and the author. Fisher's motive in studying this subject is 
to be found in classical statistical theory; that of Shannon in the 
problem of coding information; and that of the author in the 
problem of noise and message in electrical filters. Let it be 
remarked parenthetically that some of my speculations in this 
direction attach themselves to the earlier work of Kolmogo- 
roff 1 in Russia, although a considerable part of my work was 


done before my attention was called to the work of the Russian 
school, a 


The notion of the amount of inform 
naturally to a classical notion in statistical mechanics: that of 
entropy. Just as the amount of information in a system is a 
measure of its degree of organization, so the entropy of a system 


is a measure of its degree of disorganization ; and the one is 
simply the negative of the other. This point of view leads us to 
a number of considerations concerning the second law of ther- 
modynamics, and to a study of the Possibility of the so-called 
Maxwell demons. Such question 


ation attaches itself very 


nd reproduction. The third 
» that of irritability, belongs to 
1. Kolmogoroff, V.A., « Interpolation 


; > « und Extra olati tationaren Zufal- 
ligen Folgen ». Bull. Acad. Sci. U.S. S.R. See. Maik, see, 
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the domain of communication theory, and falls under the group 
of ideas we have just been discussing’. 

Thus as far back as four years ago, the group of scientists 
about Dr. Rogenblueth and myself had already become aware 
of the essential unity of the set of problems centering about 
communication, control, and statistical mechanics, whether in 
the machine or in living tissue. On the other hand, we were 
seriously hampered by the lack of unity of the literature concer- 
ning these problems, and by the absence of any common termi- 
nology, or even of a single name for the field. After much con- 
sideration, we have come to the conclusion that all the existing 
terminology has too heavy a bias to one side or another to serve 
the future development of the field as well as it should ; and as 
happens so often to scientisls, we have been forced to coin at 
least one artificial neo-Greek expression to fillthegap. Wehave 
decided to call the entire field of control and communication 
theory, whether in the machine or in the animal, by the name 

Geovetns or sleers- 


Cybernetics, which we form from the Greek xuge, 
e wish to recognize that the first 


man. In choosing this term, W 
Significant paper on feed-back mechanisms is an article on gover 
hors, which was published by Clerk Maxwell in 1868 °*, one 
that governor is derived from a Latin corruption of xugeovatns- 
We also wish to refer to the fact that the steering engines of a 
ship are indeed one of the earliest and best developed forms 
of feed-back mechanisms. 

Although the term cybernetics does not date further hack than 
the summer of 1947, we shall find it convenient to use In nefes: 
ting to earlier epochs of the development of the field. From 1942 
or thereabouts, the development of the subject went ahead on in 
several fronts. First the ideas of the joint paper of Bigelow, 
Rosenblueth, and Wiener were disseminated by Dr. Rosenblueth 
at a meeting held in New York in 1942 under the auspices of 
the Josiah Macy foundation, and devoled to problems of pa 
inhibition in the nervous system. Among those piesent at ie 
meeting was Dr. Warren McCulloch, of the Medical School © 


the University of Illinois, who had already been in touch with 


ea: Schrédinger Erwin, What is Difert Cambridge U. Press. Cambridge, Eng., 
45, 


2. Maxwell, J. C., Proc. Roy. Soc: (London). March 5, 1868. 
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Dr. Rosenblueth and myself, and who was interested in the 
study of the organization of the cortex of the brain. 

At this point there enters an element which occurs repeatedly 
in the history of cybernetics — the influence of mathematical 
logic. IfI were to choose a patron saint for cybernetics out of 
the history of science, I should have to choose Leibniz. The 
philosophy of Leibniz centers about two closely related concepts 
— that of a universal Symbolism and that of a calculus of 
reasoning. From these are descended the mathematical notation 
and the symbolic logic of the present day. Now, just as the 
calculus of arithmetic lends itself to a mechanization progres- 
sing through the abacus and the desk computing machine to the 
ultra-rapid computing machines of the present day, so the 
calculus ratiocinator of Leibniz contains the germs of the machina 
ratiocinalrix, the reasoning machine. Indeed, Leibniz himself, 
like his predecessor Pascal, was interested in the construction 
of computing machines in the metal. It is therefore nol in the 
least surprising that the same intellectual impulse which has led 
to the development of mathematical logic has at the same time 
led to the ideal or actual mechanization of processes of thought. 

A mathematical proof which we can follow is one which can 
be written in a finite number of symbols. These symbols, in 
fact, may make an appeal to the notion of infinity, but this 
appeal is one which we can sum up in a finite number of stages ; 
as in the case of mathematical induction, where we prove a 
theorem depending on a parameter n for n = 0, and also prove 
thal the case n + 1 follows from the case n, thus establishing the 
theorem for all Positive values of n. Moreover, the rules of 
Operation of our deductive r ust be finite in number, 
even though they may appear to be otherwise, through a refe- 
rence to the concept of infinity which can itself be stated in 
finite terms, In short, it has become quite evident, both to the 
Lee eden eo to the intuitionists like Weyl, that 
Ticcare sol obstitsicions ne ee theory is subject to 
sp Sonate annie : lose that limit the performance of 

8 ma + 4S We shall see laler, it is even pos- 


sible to interpret in this way the paradoxes of Cantor and of 
Russell. 
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oes Dr. Shannon took for his doctor’s thesis at the Massa- 
s Institute of Technology the application of the lechniques 
of the classical Boolean algebra of classes to the study of 
switching systems in electrical engineering. Turing, who is 
eta among those who have studied the Jogical possibi- 
Betich bio machine as an intellectual experiment, served the 
ie & ecniotent Curing the war as a WORKER in electronics; 
eee ' in charge of the program which the National Physi- 

aboratory at Teddington has undertaken for the develop- 


m ; , ; ; 
ent of computing machines of the modern type. 
the field of mathematical logic 


Another young migrant from 
He had been a stucent of Car- 


to cybernetics is Walter Pitts. 
ete oeaibacny and had also been in contact with Professor 
ay a and his school of biophysicists. Let it be remarked 

assing thal this group has contributed much to directing 
the attention of the mathematically minded to the possibilities 
of the biological sciences, although it may seem to some of us 
that they are too dominated by problems of energy and poten- 
tial and the methods of classical physics to do the best possible 
Work in the study of systems like the nervous system, which 
are very far from being closed energetically. 

Mr. Pitts had the good fortune to fall under McCulloch's influ- 
ence, and the two began lo work quite early on problems con- 
cerning the union of nerve fibres by synapses into systems 
With given over-all properties. Independently of Shannon, 
they had used the technique of mathematical logic for the dis- 
cussion of what were after all switching problems. They added 
elements which were not prominent in Shannon’s earlier work, 
although they are certainly suggested by the ideas of Turing : 
the use of the time as a parameter, the consideration of nets 
Containing cycles, and of synaptic and other delays’. 

In the summer of 1943. I met Dr. J- Lettvin of the Boston 
City Hospital, who was very much interested in matters con- 
cerning nervous mechanisms. He was a close friend of Mr. Pitts 2, 
and made me acquainted with his work. He induced Mr. Pitts 
to come out to Boston, and to make the acquaintance of Dr. Ro- 
iy, IL, 2.42, 930-265 (1936). 
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senblueth and myself. We welcomed him into our group. 
Mr. Pitts came to the Massachusetts Institute of Technology in 
the autumn of 1943, in order to work with me, and to strengthen 
hismathematical background for the study of the new science 
of cybernetics, which had by that time been fairly born, but 
not yet christened. , 
At that time Mr. Pitts was already thoroughly acquainted with 
mathematical logic and neurophysiology, but had not had the 
chance to make very many engineering contacts. In particular, 
he was not acquainted with Dr. Shannon’s work, and he had not 
had much experience of the possibilities of electronics. He was 
very much interested when I showed him examples of modern 
vacuum tubes and explained to him that these were ideal means 
for realizingin the metal the equivalents of his neuronic circuits 
and systems. From that time, it became clear to us that the 
ultra-rapid computing machine, depending as it does on conse- 
cutive switching devices, must represent almost an ideal model 
of the problems arising in the nervous system. The all-or-none 
character of the discharge of the neurons is precisely analogous 
to the single choice made in determining a digit on the binary 
scale, which more than one of us ha 
the most satisfactory basis of com 
synapse is nothing but a mech 
certain combination of outpu 


d already contemplated as 
puting machine design. The 
anism for determining whether a 
t from other selected elements 
will or will not act as an adequate stimulus for the discharge 
of the next element, and must have its precise analogue 
in the computing machine. The problem of interpreting the 
nature and varieties of memory in the animal has its parallel in 
the problem of constructing artificial memories for the machine. 

At this time, the construction of computing machines had 
proved to be more essential for the war effort than the first 
opinion of Dr. Bush might have indicated, and Was progressing 
al several centers along lines not too different from those which 
my earlier report had indicated. Harvard, Aberdee 
Ground, and the University of Pennsyly. 
structing machines, and the Institute 
Princeton and the Massachusetts Instit 
soon to enter the same field. In 
gradual progress from the mechanic 


n Proving 
ania were already con- 
of Advanced Studies at 
ute of Technology were 
this program there was a 
al assembly to the electrical 
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assembly, from the scale of ten to the scale of two, from the 
mechanical relay to the electrical relay, from humanly directed 
operation to automatically directed operation ; and in short 
each new machine more than the last was in conformity with 
the memorandum I had sent Dr. Bush. There was a continual 
going and coming of those interested in these fields. We had an 
opportunity to communicate our ideas to our colleagues, in par- 
ticular to Dr. Aiken of Harvard, Dr. von Neumann of the Insti- 
tute for Advanced Studies, and Dr. Goldstine of the Eniac and 
Edvac machines a! the University of Pennsylvania. Everywhere 
we met with a sympathetic hearing, and the vocabulary of the 
engineers soon became contaminated with the terms of the 
neurophysiologist and the psychologist. 

At this stage of the proceedings Dr. von Neumann and myself 
felt it desirable to hold a joint meeting of all those interested 
in what we now call cybernetics, and this meeting took place at 
Princeton in the late winter of 1943-1944. Engineers, physiolo- 
gists, and mathematicians were all represented. It was impossi- 
bleto have Dr. Rosenblueth among us, as he had just accepted an 
invitation to act as Head of the laboratories of physiology of the 
Instituto Nacional de Cardiologia in México, but Dr. McCulloch 
and Dr. Lorente de No of the Rockefeller Institute represented 
the physiologists. Dr. Aiken was unable to be present ; how- 
ever, Dr. Goldstine was one of a group of several computing- 
machine designers who participatedin the meeting, while Dr. von 
Neumann, Mr. Pitts, and myself were the mathematicians. The 
physiologists gave a joint presentation of cybernetic problems 
from their point of view, similarly the Computing-machine cee: 
ers presented their methods and objectives. At the end of 
the meeting, it had become clear to all that there was a sub- 
stantial common basis of ideas between the workers in the 
different fields, that people in each group could already use 
notions which had been better developed by the others, and 
that some attempt should be made to achieve a common 
vocabulary. d 

A considerable period before this, the war research group 
conducted by Dr. Warren Weaver had published a document, 
first secret and later restricted, covering the work of Mr. Bige- 
low and myself on predictors and wave filters. It was found 
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that the conditions of anti-aircraft fire did not justify the design 
of special apparatus for curvilinear prediction, but the principles 
proved to be sound and practical, and have been used by the 
government for smoothing purposes, andin several lields ofrela- 
ted work. In particular, the type of integral equation to which 
the calculus of variations problem reduces itself has been shown 
to emerge in wave-guide problems, and in many other problems 
of an applied mathematical interest. Thus in one way or ake 
ther, the end of the war saw the ideas of prediction theory and of 
the statistical approach to communication engineering already 
familiar to a large part of the statisticians and communication 
engineers of ihe United States and Greal Britain. It also saw my 
government document, now out of print, and a considerable 
number of expository papers by Levinson, Wallmann, Daniell, 
Phillips, and others 1 written to fill the gap. I myself have 
had a long mathematical expository paper under way for seve- 
ral years to put the work I have done on permanent record, but 
circumstances not completely under my control have preven- 
ted its prompt publication, Finally, after a joint meeting at the 
American Mathematical Society and the Institute of Mathemati- 
cal Statistics held in New York in the spring of 1947, and devo- 
ted to the study of Stochastic processes from a point of view 
closely allied to cybernetics, I have passed on what I have al- 
ready written of my manuscript to Professor Doob of the Univer- 
sity of Illinois, to be developed in his notation and according 
to his ideas as a book for the Mathematical Surveys series of 
the American Mathematical Society. I had already developed 
part of my work in a course of lectures in the mathematics 
department of M. I. T. in the summer of 1945. Since then, my 
old student and collaborator 2, Dr. Y, W. Lee, has returned from 
China. He is giving a course on the new methods tor the design 
of wave fillers and similar apparatus in the M. I. T. electrical 
engineering department in the fall of 1947, and has plans to 
work the material of these lectures up into a book. At the same 
time, the out-of-print sovernment document is to be reprinted. 

As I have said, Dr, Rosenblueth returned to Mexico about 
the beginning of 1944. [n the spring of 1945, I received an invi- 


1. Levinson, J. Math. and Physics (M. I. T.), 1947, 
2. Lee, Y. W., J. Math. and Physics (M.I. T.), 1932, 
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tation from the Mexican Mathematical Society to participate in 
a meeting to be held in Guadalajara that June. This invitation 
was reinforced by the Comision Instigadora y Coordinadora de 
la Investigacién Cientifica, under the leadership of Dr. Manuel 
Sandoval Vallarta, of whom I have already spoken. Dr. Rosen- 
blueth invited me to share some scientific research with him, 
and the Instituto Nacional de Cardiologia, under its director 
Dr. Ignacio Chavez, extended me its hospitality. 

I stayed some ten weeks in Mexico at that time. Dr. Rosen- 
blueth and I decided to continue a line of work which we had 
already discussed with Dr. Walter B. Cannon, who was also 
with Dr. Rosenblueth on a visit which unfortunately proved to 
be his last. ‘This work had to do with the relation between, on 
the one hand, the tonic, clonic, and phasic contractions in epi- 
lepsy, and on the other hand, the tonic spasm, beat, and fibril- 
lation of the heart. We felt that heart muscle represented an 


irritable tissue asuseful for the investigation of conduction mecha- 


nisms as nerve tissue, and furthermore, that the anastomoses 


and decussations of the heart muscle fibres presented us with 
a simpler phenomenon than the problem of tbe nervous synapse. 
We were also deeply grateful to Dr. Chivez for his unquestion- 
ing hospitality, and while it has never been the policy of the 
Instituto to restrict Dr. Rosenblueth to the investigation of the 
heart, we were grateful to have an opportunity to contribute 
lo its principal purpose. 

Ourinvestigation took two directions : the study of phenomena 
of conductivity and latency in uniform conducling media of two 
or more dimensions, and the statistical study of the conduc- 
ling properties of random nets of conducting fibres. The first 
led us to the rudiments of a theory of heart flutter, the latter 
to a certain possible understanding of fibrillation. Both lines 
of work were developed in a paper 1, published by us, and 
although in both cases our earlier results have shown the need 
of a considerable amount of revision and of supplementation, 
the work on flutter is being revised by Mr. Oliver G. Selfridge 
of the Massachusetts Institute of Technology, while the stalis- 
lical technique used in the study of heart-muscle nets has been 


1. Wiener, N.. Rosenblueth, A. Conduction of Impulses in Cardine Muscle ». 
= , hd , 
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extended to the treatment of neuronal nets by Mr. Waller Pitts, 
now a Fellow ofthe John Simon Guggenheim Foundation. The 
experimental work is being carried on by Dr. Rosenblueth with 
the aid of Dr. F. Garcia Ramos of the Instituto Nacional de Car- 
diologia and the Mexican Army Medical School. 

At the Guadalajara meeting of the Mexican Mathematical So- 
ciety, Dr. Rosenblueth and Ipresented some of our results. We 
had already come to the conclusion that our earlier plans of col- 
laboration had shown themselves to be practicable. We werefor- 
{unate enough to have a chance to present our results toa larger 
audience. Inthe spring of 1946, Dr. McCulloch had made arrange- 
ments with the Josiah Macy foundation for thefirst of a series of 
meelings to be held in New York and to be devoted to the pro- 
blems of feed back. These meetings have been conducted in the 
traditional Macy way, worked out most efficiently by Dr. Frank 
Fremont-Smith, who organized them on behalf of the Founda- 
tion. The idea has been to get together a group of modest size, 
not exceeding some twenty in number, of workers in various 
related fields, and to hold them together for two successive days 
in all-day series of informal papers, discussions, and meals 
together, until they had had the Opportunity to thresh out their 
differences and to make Progress in thinking 


along the same 
lines. The nucleus of our meetings h 


as been the group that had 


and sociological 
ve co-opted into the group a 
number of leading Psychologists, sociologists, and anthropolo- 


chologists had indeed been 


he psychology of the past has 
an the physiology of the organs 
ight of the body of ideas which 
psychology concerns the phy- 
Y specialized cortical areas con- 
cial sense. 
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nize a square as a square, irrespective of ils position, its size, 
and its orientation ? To assist us in such matters, and to inform 
them of whatever use might be made of our concepts for their 
assistance, we had among us such psychologists as Professor 
Kliiver of the University of Chicago, the late Dr. Kurt Lewin of 
the Massachusetts Institute of Technology, and Dr. M. Ericsson 
of New York. 

As to sociology and anthropology, itis manifest that the im- 
portance of information and communication as mechanisms of 
organization proceeds beyond the individualinto the community. 
On the one hand, it is completely impossible to understand social 
communities such as those of ants without a thorough investiga- 
tion of their means of communication, and we were fortunate 
enough to have the aid of Dr. Schierda in this matter. For the 
similar problems of human organization, we sought help from 
the anthropologists Drs. Bateson and Margaret Mead ; while 
Dr. Morgenstern of the Institute of Advanced Studies was our 
adviser in the significant field of social organization belonging 
to economic theory. His very important joint book on games 
with Dr. von Neumann, by the way, represents a most interes- 
ting study of social organization from the point of view of 
, although distinct from, the subject 
matter of cybernetics. Dr. Lewin and others represented the 
newer raul on the theory of opinion sampling and the prac- 
tice of opinion making, and Dr. F.C. ) Northrup was: totes 
rested in assaying the philosophical significance of our work. 

This does not purport lo be a complete list of our group. 
We also enlarged the group to contain more engineers and ma- 
thematicians suchas Bigelow and Savage pmore neuroanatomists 
and neurophysiologists such as von Bonin aod Lloyd, and so 
on. Our first meeting, held in the spring of 1946, was largely 
devoted to didactic papers by those of us who had been present 
at the Princeton meeting and to & general assessment of the im- 
portance of the field by all present. It ae the sense of the 
meeting that the ideas behind cybernetics were sufficiently 
important and interesting to those present to warrant a conti- 
nuation of our meetings at intervals of six months; and that 
before the next full meeting, We should have a small meeting 
for the benefil of the less mathematically trained to explain to 


methods closely related to 
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them in as simple language as possible the nature of the mathe- 
matical concepts involved. 

In the summer of 1946, I returned to Mexico with the support 
of the Rockefeller Foundation and thehospitality of'the Instituto 
Nacional de Cardiologia to continue the collaboration between 
Dr. Rosenblueth and myself. This time we decided to take a 
nervous problem directly from the topic of feed-back, and to see 
what we could do with it experimentally. We chcse the cat as 
our experimental animal, and the quadriceps extensor femoris 
as the muscle to study. We cut the attachment of the muscle, 
fixed itto a lever under known tension, and recorded its contrac- 
tions isometrically or isotonically. Wealso used an oscillograph 
to record the simultaneous electrical changes in the muscle 
itself. We worked chiefly with cats, first decerebrated under 
ether anesthesia and later made spinal by a thoracic transection 
ofthecord. In many cases strychnine was used to increase the 
reflex responses. The muscle was loaded to the point where a 
tap would set it into a periodic pattern of contraction, which is 
called clonus in the language of the physiologist. We observed 
this pattern of contraction, paying attention to the physiological 
condition of the cat, the load on the muscle, the frequency of 
oscillation, the base-level of the oscillation, and its amplitude. 
These we tried to analyze as we should analyze a mechanical 
or electrical system exhibiting the same pattern of hunting. We 
employed, for example, the methods of McColl’s book on servo- 
mechanisms. This isnot the place to discuss the full significance 
of our results, which we are how repeating and preparing to 
write up for publication. However, the following statements are 
either established or very probable : that the frequency of clonic 
oscillation is much less sensitive to changes of the loading con- 
ditions than we had expected ; and that it is much more nearly 
determined by the constants of the closed arc efferent-nerve- 
muscle-kinaesthetic-end -body-afferent-nerve-central-synapse- 
efferent-nervethan by anything else. This circuit is not even 
approximately a circuit of linear operators if we take as our 
base of linearity the number of impulses transmitted by the 
efferent nerve per second, but seems to become much more 
nearly so if we replace the number of impulses by its logarithm. 
This corresponds to the fact that the form of the envelope of 
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stimulation of the efferent nerve is not nearly sinusoidal, but 
that the logarithm of this curve is much more nearly sinusoidal; 
while in a linear oscillating system with constant energy level, 
the form of the curve of stimulation must be sinusoidal in all 
except a set of cases of zero probability. Again, the notions of 
facilitation and inhibition are much more nearly multiplicative 
than additive in nature. For example, a complete inhibition 
means a multiplication by zero, and a partial inhibition means 
a multiplication by a small quantity. It is these notions of 
inhibition and facilitation which have been used ? in the dis- 
cussion of the reflex arc. Furthermore, the synapse isa coin- 
cidence-recorder, and the outgoing fibre is only stimulated if 
the number of incoming impulses in a small summation-time 
exceeds a certain threshold. If this threshold is low enough in 
comparison with the full number of incoming synapses, the 
synaptic mechanism serves to multiply probabilities, and thal it 
can be even an approximately linear link is only possible in a 
logarithmic system. This approximate logaritamicily of the 
synapse mechanism is certainly allied 16 the aponesiouts 
logarithmicily of the Weber-Fechner law of sensation intensity, 
even though this law is only a first approximation. 

The most striking point is that on this logarithmic basis, and 
with data obtained from the conduction of single pulses through 
the various elements of the neuromuscular arc, we were able to 
obtain very fair approximations to the actual periods of clonic 
vibration, using the technique already developed by the servo- 
engineers for the determination of the frequencies of hunting 
oscillations in feed-back systems which have broken down. We 
obtained theoretical oscillations of about 13.9 per second, in 
pserved oscillations varied between frequen- 
ally remained within a range varying 
Under the circumstances, this 
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agreement is excellent. 


The frequency ofclonus is nol the only important phenomenon 


which we may observe : there is also a relatively slow change in 
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phenomena are certainly by no means linear. However, suffi- 
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ciently slow changes in the constants of a linear oscillating sys- 
tem may be treated to a first approximation as though they 
were infinitely slow, and as though over each part of the 
oscillation the system behaved as it would if ifs parameters 
were those belonging to it at the time. This is the method 
known in other branches of physics as that of secular perturba- 
tions. It may be used to study the problems of base-level and 
amplitude of clonus. While this work has not yet been com- 
pleted it is clear that it is both possible and promising. There 
is a strong suggestion that though the timing of the main are in 
clonus proves it to be a two-neuron arc, the amplification of 
impulses in’this arc is variable in one and perhaps in more 
points, and that some part of this amplification may be affected 
by slow, multi-neuron processes which run much higher in the 
central nervous system than the spinal chain primarily respon- 
sible for the timing of clonus. This variable amplification may 
be affected by the general level of central activily, by the use 
of strychnine or of anesthetics, by decerebration, and by many 
other causes. 

These were the main results presented by Dr. Rosenblueth 
and myself at the Macy meeting held in the autumn of 1946, and in 
ameeting of the New York Academy of Sciences held at the same 
time for the purpose of diffusing the notions of cybernetics over 
a larger public. While we were pleased with our results, 
and fully convinced of the general practicability of work in this 
direction, we felt nevertheless that the time of our collaboration 
had been too brief, and that our work had been done under too 
much pressure to make it desirable to publish without further 
experimental confirmation. This confirmation — which natu- 
rally might amount to a refutation — we are now seeking in the 
summer and autumn of 1947, 

The Rockefeller Foundation had already given Dr. Rosenblueth 
a grant for the equipment of a new laboratory building at the 
Instituto Nacional de Cardiologia. We felt that the lime was 
now ripe for us to go jointly to them — that is, to Dr. Warren 
Weaver, in charge of the department of physical sciences, and 
to Dr. Robert Morison, in charge of the department of medical 
sciences — to establish the basis of a long-time scientific collabo- 
ration,in order to carry on our program at a more leisurely and 


INTRODUCTION 31 


healthy pace. In this we were enthusiastically backed by our 
respective institutions. Dr. George Harrison, Dean of Sciences, 
was the chief representative of the Massachusetts Institute of 
Technology during these negotiations, while Dr. Ignacio Chavez 
spoke for his institution, the Instituto Nacional de Cardiologia. 
During the negotiations, it became clear that the laboratory 
center of the joint activily should be at the Instituto, both in 
order to avoid the duplication of laboratory equipment and to 
further the very real interest the Rockefeller Foundation has 
shown in the establishment ofscientific centers in Latin America. 
The plan finally adopted was for five years, during which Ishould 


spend six months of every other year at the Instituto, while 


Dr. Rosenblueth would spend six months of the intervening years 
at the Institute. The time at the Instituto is to be devoted to the 
obtaining and elucidation of experimental data pertaining to 
cybernetics, while the intermediate years are to be devoted to 
more theoretical research, and above all, to the very difficult 
problem of devising, for people wishing to go into this new 
field, a scheme of training which will secure for them both the 
necessary mathematical, physical, and engineering background, 
and the proper acquaintance with biological, psychological, 
and medical techniques. 

In the spring of 1947, Dr. McCulloch and Mr. Pitts did a piece of 
work of considerable cybernetic importance. Dr. McCulloch 
had been given the problem of designing an apparatus to enable 
the blind to read the printed page by ear. The production of 
variable tones by type through the agency of a photocell is an 
old story, and can be effected by any number of methods ; the 
difficult point is to make the pattern of the sound substantially 
attern of the letters is given, whatever the 
ogue of the problem of the percep- 
tion of form, of Gestalt, which allows us to recognize a square 
arge number of changes of size and 
device involved a selective 


the same when the p 
size. This is a definite anal 


as a square through a 1 
of orientation. Dr. McCulloch's im 
reading of the type-imprint for a set of different magnifica- 
tions. Such a selective reading can be perfomed automatically 
This scanning, to allow a compari- 
a given standard figure of fixed but 
h I had already suggested at one 


as a scanning process. 
son between a figure and 
different size, was a device whic 
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of the Macy meetings. A diagram of the apparatus by which 
the selective reading was done came to the attention of Dr. yon 
Bonin, who immediately asked, « Is this a diagram of the 
fourth layer of the visual cortex of the brain ? » Acting on this 
suggestion, Dr. McCulloch, with the assistance of Mr. Pitts, 
produced a theory tying up the anatomy and the physiology of 
the visual certex, and in this theory the operation of scanning 
over a set of transformations plays an important part. ‘This was 
presented in the spring of 1947 both at the Macy meeting and 
at a meeting of the New York Academy of Sciences. Finally, 
this scanning process involves a certain periodic time, which 
correspond$ to what we call the « time of sweep » in ordinary 
television. There are various anatomic clues to this time in 
the length of the chain of conseculive synapses necessary lo 
run around one cycle of performance. These yield a time of 
the order of a tenth ofa second for a complete performance of 
the cycle of operations, and this is the approximate period of 
the so-called « alpha rhythm » of the brain. Finally, the alpha 
rhythm, on quite other evidence, has already been conjectured 
to be of visual origin, and to be important in the process of form 
perception. 

In the spring of 1947, I received an invitation to participate 
in a mathematical conference in Nancy on problems arising from 
harmonic analysis. [| accepted, and on my voyage there and 
back spent a total of three weeks in England, chiefly as a guest 
of my old friend Professor J. B. S. Haldane. { hadan excellent 
chance lo meet most of those doing work on ultra-rapid com- 
puting machines, especially at Manchester and at the National 
Physical Laboratories at Teddington, and above all to talk over 
the fundamental ideas of cybernetics with Mr. Turing at Tedding- 
ton. I also visited the Psychological Laboratory at Cambridge, 
and had a very good chance to discuss the work that Professor 
F. C. Bartlett and his staff were doing on the human element in 
control processes involving such an element. I found the inte- 
rest in cybernetics about as great and well informed in England 
as in the United States, and the engineering work excellent, 
though of course limited by the smaller funds available. I found 
much inlerest and understanding of its 


possibility in many 
quarters, and Professors Haldane, H. I 


+evy and Bernal certainly 
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regarded it as one of the most urgent problems on the agenda 
of science and scientific philosophy. I did not find, however, 
that as much progress had been made in unifying the subject and 
in pulling the various threads of research together as we had 
made at home in the States. 

In France, the meeting at Nancy on harmonic analysis con- 
tained a number of papers uniting statistical ideas and ideas 
from communication engineering in a manner wholely in confor- 
mity with the point of view of cybernetics. Here 1 must mention 
mes of M. Blanc-Lapierre and M. Loéve. | I 
able interest in the subject on the part of 
physical chemists, particu- 


especially the na 
found also a consider: 
mathematicians, physiologists, and 
larly with regard to its thermodynamic aspects in so far as they 
touch the more general problem of the nature of life itself. 
Indeed, I had discussed thal subject in Boston before my depar- 
ture with Professor Szent-Gydorgi, the Hungarian biochemist, and 
had found his ideas concordant with my own. 

One event during my French visit is particularly worth 


while noting here. My colleague, Professor G. de Santillana of 


M.I. T, introduced me to M. Freymann, of the firm of Her- 
me the present book. I am 


mann et Cie, and he requested of 1 
particularly glad to receive his invitation as M Freymann is a 
Mexican, and the writing of the present book, as well as a good 
deal of the research leading up to it, have been done in Mexico. 
AsI have already hinted, one of the directions of work which 
the realm of ideas of the Macy meetings has suggested concerns 
the importance of the notion and the technique of communica- 
tion in the social system. It is certainly true that the social 
system is an organization like the individual ; that it is bound 
together by a system of communication ; and that it has a dyna- 
mics, in which circular processes of a feed back nature play an 
important part. This is true, both in the general fields of anthro- 
pology and of sociology, and in the more specific field of eco- 
nomics ; and the very important work, which we have already 
mentioned, of von Neumann and Morgenstern on the theory of 
range of ideas. On this basis, Drs. Gre- 
ad have urged me, in view of the 
e sociological and economic pro- 
to devote a large part 


games, enters into this 
gory Bateson and Margaret Me 
intensely pressing nature of th 
blems of the present age of confusion, 


34 CYBERNETICS 


of my energies to the discussion of this side of cybernetics. 

Much as I sympathize with their sense of the urgency of the 
situation, and much as I hope that they and other competent 
workers will take up problems of this sort which I shall dis- 
cuss in a later chapter of this book, 1 can share neither their 
feeling that this field has the first claim on my atlenlion, nor 
their hopefulness that sufficient progress can be registered in this 
direction to have an appreciable therapeulic effect in the present 
diseases of society. To begin with, the main quantities affecting 
society are not only statistical, but the runs of statistics on which 
they are based are excessively short. There is no great use in 
lumping urider one head the economics of steel industry before 
and after the introduction of the Bessemer process. nor in com- 
paring the statistics of rubber production before and after the 
burgeoning of the automobile industry and the cultivation of 
Hevea in Malaya. Neither is there any important point in run- 
ning statistics of the incidents of venereal disease ina single table 
which covers both the period before and that after the introduc- 
tion of salvarsan, unless for the specific purpose of studying the 
effectiveness of this drug. For a good stalislic of society, we 
need long runs under essentially constant conditions, just as for 
a good resolution of light we need a lens with a large aperture. 
The effective aperture of a lens is not appreciably increased by 
augmenting its nominal aperture, unless the lens is made of a 
material so homogeneous that the delay of light in different parts 
of the lens conforms to the proper designed amount by less than a 
small part of a wavelength. Similarly, the advantage of long runs 
of statistics under widely varying conditions is specious and spu- 
rious. Thus the human sciences are very poor testing-grounds 
for a new mathematical technique : as poor as the statistical 
mechanics of a gas would be to a being of the order of size of a 
molecule, to whom the fluctuations which we ignore from a 
larger standpoint would be precisely the matters of greatest 
interest. Moreover, in the absence of reasonably safe routine 
numerical techniques, the element of the judgment of the expert 
in determining the estimates to be made of sociological, anthro- 
pological, and economic quantities, is so great that it is no field 
for a newcomer who has not yet had the bulk of experience 
which goes lo make up the expert. I may remark parenthetically 
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that the modern apparatus of the theory of small samples, once 
it goes beyond the determination of its own specially defined 
parameters and becomes a method for positive statistical infe- 
rence in new cases, does not inspire me with any confidence, 
unless it is applied by a statistician by whom the main elements 
of the dynamics of the situation are either explicitly known or 
implicitly felt. 

I have just spoken of a field in which my expectations of 
cybernetics are definitely tempered by an understanding of the 
limitations of the data which we may hope to oblain. There 
are two other fields where I ultimately hope to accomplish 
something practical wilh the aid of cybernetic ideas, but in 
which this hope must wait on further developments. One of 
these is the matter of prostheses for lost or paralyzed limbs. 
As we have seen in the discussion of Gestalt, the ideas of com- 
ng have already been applied by McCulloch 
ement of lost senses, in the construc- 
able the blind to read print by hea- 
ting. Here the instrument suggested by McCulloch takes over 
quite explicitly some of the functions nol only of the eye but of 
There is a manifest possibility of doing 
ase of artificial limbs. The loss of a 
ly the loss of the purely passive 


support of the missing segment or ils value as mechanical exten- 
sion of the stump, and the loss of the contractile power of ils 
niuscles, but implies as well the loss of all culaneous and kinaes- 
thetic sensations originating init. The first two losses are what 
the artificial-limbmaker now tries to replace. The third has so 
far been beyond his scope. In the case of a simple peg-leg this 
is not important: the rod that replaces the missing limb has no 
degrees of freedom of ils own, and the kinaesihetic mechanism of 
the stump is fully adequate to report its own position and velo- 
city. This is not the case with the articulated limb with a mo- 
_ bile knee and ankle, thrown ahead by the patient with the aid 
of his remaining musculature. He has no adequale report of 
their position and motion, and this interferes with his sureness 
of step on an irregular terrain. There does not seem to be any 


insuperable difficully in equipping the arlificial joints and the 
sole of the artificial foot with strain or pressure gauges, which 


munication engineeril 
to the problem of the replac 
tion of an instrument to en 


the visual cortex. 
something similar in the ¢ 
segment of limb implies not on 
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are to register electrically or otherwise, say through vibrators, 
on intact areas of skin. The present artificial limb removes 
some of the paralysis caused by the amputation, but leaves the 
ataxia. With the use of proper receptors, much of this alaxia 
should disappear as well, and the patient should be able to 
learn reflexes such as those we all use in driving a car, which 
should enable him to step out witha much surer gait. What 
we have said about the leg should apply with even more force 
to the arm, where the figure of the manikin familiar to all 
readers of books of neurology shows that the sensory loss in 
an amputation of the thumb alone is considerably greater than 
the sensory loss even in a hip-joint ampulation. 


[have made an attempt to report these considerations to the 
proper authorities, but up to now I have not been able to accom- 
plish much. Idonot know whether the same ideas have already 
emanated from other sources, nor whether they have been tried 
out and found technically impracticable. In case they have nol 
yet received a thorough practical consideration, they should 
receive one in the immediate future. 


Let me now come to another point whichih elieve to meril at- 
tention. It has long been clear to me that the modern ultra-rapid 
computing machine was in principle an ideal central nervous 
system toan apparatus for aulomatic control; and thatitsinputand 
output need not be in the form of numbers or diagrams, but might 
very well be, respectively, the readings of artificial sense-organs 
such as photo-electric cells or thermometers, and the performance 
of molorsorsolenoids. With the aid of strain-gauges or similar 
agencies to read the performance of these motor organs and to 
report, to « feed back », to the central control system as an arti- 
ficial kinaesthetic sense, we are already ina position to construct 
artificial machines of almost any degree of elaborateness of 
performance. Long before Nagasaki and the public awareness 
of the atomic bomb, it had occurred to me that we were here 
in the presence of another social potentiality of unheard-of im- 
portance for good and for evil. The automatic factory, the as- 
sembly-line without human agents, are only so far ahead of us 
as is limited by our willingness to put such a degree of effort 
into their engineering as was spent, for example, in the deve- 
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lopment of the technique of radar in the second world 
war 7. 

1 have said that this new development has unbounded possi- 
bilities for good and for evil. For one thing, it makes the 
metaphorical dominance of the machines, as imagined by Samuel 
Butler, a most immediate and non-metaphorical problem. It 
gives the human race a new and most effective collection of 
mechanical slaves to perform its labor. Such mechanical labor 
has most of the economic properties of slave labor, although, 
it does not involve the direct demoralizing 
effects of human cruelty. However, any labor that accepts the 
conditions of competition with slave labor, accepts the conditions 
and is essentially slave labor. The key word of 
It may very well be a good thing 
for humanity to have the machine remove from it the need of 
menial and disagreeable tasks ; oritmaynot. Ido not know. It 
cannot be good for these new potentialities to be assessed in 
the terms of the market, of the money they save ; and it is 
precisely the terms of the open market, the « fifth freedom», 
that have become the shibboleth of the sector of American opinion 
represented by the National Association of Manufacturers and the 
Saturday Evening Post. 1 say American opinion, for as an 
American, I know it best, but the hucksters recognize no na- 
tional boundary. 

Perhaps I may clarify 


unlike slave labor, 


of slave labor, 
this statement is competition. 


the historical background of the present 
situation if Isay that the first industrial revolution, the revolution 
of the « dark satanic mills », was the devaluation of the human 
arm by the competition of machinery. There is no rate of pay 
at which a United States pick-and shovel laborer can live which 
is low enough to compete with the work of a steam shovel as an 
excavator. The modern industrial revolution is similarly bound 


to devalue the human brain at least in ils simpler and more 
routine decisions. Of course, just as the skilled carpenter, the 
skilled mechanic, the skilled dressmaker have in some degree 
survived the first industrial revolution, so the skilled scientist 
and the skilled administrator may survive the second. However, 
taking the second revolution as accomplished, the average human 


1. Fortune, 1946. 
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being of mediocre attainments or less has nothing to sell that 
it is worth anyones money to buy. 

The answer, of course, is to have a sociely based on human 
values other than buying or selling. ‘To arrive at this society 
we need a good deal of planning and a good deal of struggle -- 
which, if the best comes to the best, may be on the plane of 
ideas, and otherwise — who knows ? [ thus felt it my duty to 
pass on my information and understanding of the position to 
those who have an active interest in the conditions and the future 
of labor — that is to the labor unions. I did manage to make 
contact with one or two persons high up in the C. I. (., and 
from them I received a very intelligent and sympathetic hearing. 
Further than these individuals neither I norany of them was able 
to yo. It was their opinion, as it had been my previous obser- 
valion and information, both in the United States and in England, 
that the labor unions and the labor movement are in the hands 
of a highly limited personnel, thoroughly well trained in the 
specialized problems of shop-stewardship and disputes concer- 
ning wages and conditions of work; and totally unprepared to 
enter into the larger political, technical, sociological, and 
economic questions which concern the very existence of labor. 
The reasons for this are easy enough to see: the labor union 
official generally comes from the exacting life of a workman into 
the exacling life of an administrator without any opportunity 
for a broader training; and for those who have this training, 
a union career is not generally inviting ; nor, quile naturally, 
are the unions receptive to such people. 

Those of us who have contributed to the new science of 
cybernelics thus stand in a moral posilion which is, to say the 
least, not very comfortable. We have contributed to the inilia- 
tion of a new science which, as I have said, embraces tech- 
nical developments with great possibilities for good and for 
evil. We can only hand it over into the world that exists about 
us, and this is the world of Belsen and Hiroshima. We do not 
even have the choice of suppressing these new technical deve- 
lopments. They belong to the age, and the most any of us can do 
by suppression is to put the development of the subject into the 
hands of the most irresponsible and most venal of our engineers. 
The best we can do is to see thata large public understands 
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the trend and the bearing of the present work, and to confine 
our personal efforts to those fields, such as physiology and 
psychology, most remote from war and exploitation. As we 
have seen, there are those who hope that the good of a better 
understanding of man and sociely which is offered by this new 
field of work may anticipate and oulweigh the incidental 
contribution we are making to the concentration of power 
(which is always concentrated, by its very conditions of existence, 
in the hands of the most unscrupulous). 1 write in 1947, and I 
am compelled to say that itis a very slight hope. 

The author wishes to express his gratitude lo Mr. Walter Pitts, 
Mr. Oliver Selfridge, Mr. Georges Dubé and Mr. Frederic Webster 
for aid in correcting the manuscript and preparing the material 
for publication. 
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CHAPTER I 


NEWTONIAN AND BERGSONIAN TIME 


There is a little hymn or song familiar lo every German child. 
It goes : 
« Weisst wie viele Sterne stehen 
In dem blauen Himmelszelt 2 
Weisst, wie viele Wolken gehen 
Weit hiniitber alle Welt ? 
Gott der Herr hat sie gezihlet 
Dass ihm auch nicht eines fehlet 
Von der ganzen grossen Zahl », 


In English this says : « Knowest thou how many stars stand 
in the blue tent of heaven ? Knowest thou how many clouds 
pass far over the whole world? The Lord God hath counted 
them, that not one of the whole great number be lacking ». 

This little song is an interesting theme for the philosopher 
and the historian of science, in that it puts s 
sciences which have the one simil 
ens above us, but which in 
extreme contrast. 
while meteorolog 
the name. 


ide by side two 
arity of dealing with the heay- 
almost every other respect offer an 
Astronomy is the oldest of the sciences, 
'y is among the youngest to begin to deserve 
The more familiar astronomical phenomena can be 
predicted for many centuries, while a precise prediction of 


tomorrow's weather is generally not easy and in many places 
very crude indeed, 


To go back to the 


poem, the answer to the first question is 
that, within jimits, w 


e do know how many stars there are. In 
the first place, apart from minor uncertainties concerning some 
of the double and variable stars, a staris a definite object, emi- 
nently suitable for counting and cataloguing ; and if a human 
Durchmusterung of the stars — as we call these catalogues — 
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stops short for stars less intense than a certain magnitude, 
there is nothing too repugnant to us in the idea of adivine Durch- 
musterung going much further. 

On the other hand, if you were to ask the meteorologist to 
give youa similar Durchmuslerung of the clouds, he might laugh 
in your face, or he might patiently explain that in all the Jan- 
guage of meteorology, there is no such thing as a cloud, defined 
as an object with a quasi-permanent identity ; and that if there 
Were, he neither possesses the facilities to count them, nor is he 
in fact interested in counting them. A topologically inclined me- 
teorologist might perhaps define a cloud as a connected region 
of space in which the density of the part of the water content 
in the solid or liquid state exceeds a certain amount, but this 
definition would not be of the slightest value to anyone, and 
would at most represent an extremely transilory state. What 
really concerns the meteorologist is some such statistical state- 
ment as, « Boston. January 17, 1950. Sky 38 op overcast. Cirro- 


cumulus ». 


There is of course a br 
what may be called cosmic 


anch of astronomy which deals with 
meteorology : the study of galaxies 


and nebulae and star-clusters and their statistics, as pursued for 
example by Chandrashekhta, but this is a very young hranch 
of astronomy, younger than meteorology itself, andis something 
outside the tradition of classical astronomy. This tradition, 
apart from its purely classificatory, Durchmusterung aspects, 
ather with the solarsystem than with 
Itis the astronomy of the solar 
associated with the names of 
and Newton, and which was the 


was originally concerned r 
the world of the fixed stars. 
system which is that chiefly 
Copernicus, Kepler, Galileo, 


wet-nurse of modern physics. : 
It is indeed an ideally simple science. Even before the exis- 


tence of any adequate dynamical theory. Evens’ far back as the 
Babylonians, it was realized that eclipses occurred in regular pre- 
dictable cycles, extending backwards and forwards over time. It 
was fanlizetl that time itself could better be measured by the 
motion of the stars in their courses than in any other way. The 
pattern for all events in the solar system was the revolution of a 
wheel or a series of wheels, whether in the form of the Ptolemaic 
theory of epicycles or the Copernican theory of orbits, and in any 
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such theory, the future after a fashion repeats the past. The 
music of the spheres is a palindrome, and the book of astro- 
nomy reads the same backwards as forwards. There is no differ- 
ence save of initial positions and directions between the motion 


of an orrery turned forwards and one run in reverse. Finally, 
when all this was reduced by Newton to a formal set of posty- 
lates and a closed mechanics, the fundamental laws of this 
mechanics were unaltered by the transformation of the time- 
variable f into its negative. 

Thus if we were lo take a motion-picture of the planets, speed- 
ed up to show a perceptible picture ofactivity, and were lo run 
the film ba¢kward, it would still be a possible picture of pla- 
nets conforming to the Newtonian mechanics. On the other 
hand, if we were to take a motion-picture photograph of the 
turbulence of the clouds in a thunderhead, and reverse it, it 
would look allogether wrong. We should see down-dralts 
where we expect up-drafts, turbulence growing coarser in tex- 
ture, lightning preceding instead of following the changes of 
cloud which usually precede it, and so on indefinitely. 

What is the difference between the astronomical and_the 
meteorological situation which brings about all these differences, 
and in particular the difference between the apparent reversibi- 
lity of astronomical time and the apparent irreversibility of 
meteorological time? In the first place, the meteorological 
system is one involving a vast number of approximately equal 
particles, some of them very closely coupled to one another, 
while the astronomical system of the solar universe contains 
only a relatively small number of particles, greatly diverse in 
size, and coupled with one another in a sufficiently loose way 
that the second order coupling effects do not change the general 
aspect of the picture we observe, and the very high order coup- 
ling effects are completely negiigible. The planets move under 
conditions more favorable to the isolation of a certain limited 
Set of forces than those of any physical experiment we can set 
up in the laboratory. Compared with the distances between 
them, the planets, and even the sun, are very nearly points, 
Compared with the elastic and plastic deformations they suffer, 
the planets are either very nearly rigid bodies ; or where they 
are not, theirinternal forces are at any rate of a relatively slight 
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significance where the relative motion of their centres is con- 


cerned. The space in which they move is almost perfectly free 
in their mutual attraction, their 


from impeding matter ; and 
dered to lie at [heir centres and 


masses may be very nearly consi 
to be‘constant. The departure of the law of gravity from the 
inverse square law is most minute. The positions, velocilies, 
and masses of the bodies of the solar system are extremely 
well known at any time, and the computation of their future 
and past positions, while not easy in detail, is easy and pre- 
On the other hand, in meteorology, the num- 
ber of particles concerned is so enormous that an accurate 
récord of their initial positions and velocities is ulterly impos- 
sible ; and if this record were actually made, and their fulure 
Positions and velocities computed, we should have nothing but 
an impenetrable mass of figures which would need a radical 
reinterpretation before it could be of any service to us. The 
terms « cloud », « temperature », « turbulence », etc., are all 
terms referring nol lo one single physical situation, but to a 
distribution of possible siluations of which only one actual case 
is realized. If all the readings ofall the meteorological stations 
on earth were simultaneously taken, they would not give a bil- 
lionth part of the data necessary to characterize the actual state 
of theatmosphere from a Newtonian point ofview. They would 
only give certain constanls consistent with an infinily of diffe- 
rent atmospheres, and at most, together with bextaln a priori 
assumptions, capable of giving as a probability distribution, a 
measure, over the set of possible atmospheres. Using the New- 
tonian laws, or any olher system of causal laws whatever, all 
that we can predict at any fulure time is a probability distribu- 
tion of the constants of the system, and even this predictabilily 
fades out with the increase of lime. 

Now, even in a Newtonian system, in which time is perfectly 
reversible, questions of probability and prediction lead to 
answers asymmetrical as between past and future, because the 
questions to which they are answers are asymmetrical. if I set 
up a physical experiment, I bring the system I am considering 
from the past into the present in such a way that I fix certain 
quantities, and have a reasonable righl to assume that cerlain 
other quantities have known stlalistical distributions. I then 


Cise in principle. 
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observe the statistical distribution of results after a given time. 
This is not a process which I can reverse. In order to do so, 
it would be necessary to pick oul a fair distribution of systems 
which, without intervention on our part, would end up within 
certain slalistical limits, and find out what the antecedent 
conditions were a given time ago. However, for a system 
starting from an unknown position to end up in any tightly 
defined statistical range is so rare an occurrence that we may 
regard it as a miracle, and we cannot base our experimental 
technique on awailing and counting miracles. In short, we are 
directed in time, and our relation to the future is different from 
our relation to the past. All our questions are conditioned by 
this asymmetry, and all our answers to these questions are 
equally conditioned by it. 


A very interesting astronomical question concerning the 


direction of time comes up in connection with the time of 


astrophysics, in which we are observing remote heavenly bodies 
in a single observation, and in which there seems to be no 
unidirectionalness in the nature of our experiment. Why then 
does the unidirectional thermodynamics which is based on 
experimental terrestrial observations stand us in such good 
stead in astrophysics ? The answer is interesling, and not too 
obvious. Our observations of the Stars are through the agency 
of light, of rays or particles emerging from the observed object, 
and perceived by us. We can perceive incoming light, but can 
not perceive outgoing light ; or at least the perception of 
outgoing light is not 


achieved by an exper 
direct 


iment as simple and 
as that of incoming light. 


; In the Perception of incoming 
light, we end up with the eye or a Photographic plate. We 


condition these for the reception of images by putting them in 
a state of insulation for Some time past : we dark-condilion 
the eye to avoid after-images, and Wwe wrap our plates in black 
paper to prevent halation, It is clear that only such an eye and 
only such plates are any use to us: if we were given to pre-images 
we might as well be blind, and if we had to pul our plates in 
black paper after we use them and develop them before using, 
photography would be a very difficult art indeed. This being 
the case, we can see those stars radiating to us and to the whole 
world ; while if there are any stars whose evolution is in the 
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reverse direction, they will attract radiation from the whole 
heavens, and even this attraction from us willnot be perceptible 
in any way, in view of the fact that we already know our own 
past but not our future. Thus the part of the universe which 
we see must have its past-fulure relations, as far as the emission 
of radiation is concerned, concordant with our own. The very 
fact that weseea slar means that ils thermodynamics is like our 
own. 

Indeed, it is a very inleresting intellectual experiment to make 
the fantasy of an intelligent being whose time should run the 
To such a being all communication 
with us would be impossible. Any signal he might send would 
reach us with a logical stream of consequents from his point of 
view, antecedents from ours. These antecedents would already 
, and would have served to us as the 
], without presupposing an 
If he drew us a square, we 


other way to our own. 


be in our experience 
natural explanation of his signa 
intelligent being to have sent it. 
should see the remains of his figure as its precursors, and it 
Would seem to be the curious crystallization — always perfectly 
explainable — of these remains. Its meaning would seem to 
be as fortuitous as the faces we read into mountains and cliffs. 
The drawing of the square would appear lo us as a catastrophe 
— sudden indeed, but explainable by natural laws — by which 
that square would cease to exist. Our counterpart would have 
exactly similar ideas concerning Us. Within any world w th which 
we can communicate, the direction of time ts uniform. 

To return to the contrast between Newtonian astronomy and 
Meteorology : most sciences lie in an intermediate position, but 
Most are rather nearer to meteorology than to astronomy. 
Even astronomy, as We have a cosmic meteoro- 


seen, contains 
logy. Itcontains as well that extremely interesting field studied 
by Sir George Darwin, and known as 


the theory of tidal evolution. 
We have said that we can treat the relative movements of the 
sun and the planets as the mov 


ements of rigid bodies, but this is 
not quite the case. The earth, for example, is nearly surrounded 
by oceans. The water 


nearer the moon than the center of the 
earth is more strongly attracted to the moon than the solid part 
of the earth, and the waler 


on the other side is less strongly 
attracted. This relatively slight effect pulls the water into two 
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hills, one under the moon and one opposite lo the moon. In 
a perfectly liquid sphere, these hills could follow the moon 
around the earth with no great dispersal of energy, and conse- 
quently would remain almost precisely under the moon and 
opposite to the moon. They would consequently have a pullon 
the moon which would not greatly influence the angular position 
of the moon in the heavens. However, the tidal wave they 
produce on the earth gets tangled up and delayed on coasts and 
in shallow seas such as the Behring Sea and the Irish Sea. It 
consequently lags behind the position of the moon, and the forces 
producing this are largely turbulent, dissipative forces of a 
character much like the forces met in meteorology, and need a 
statistical treatment. Indeed, oceanography may be called 
the meteorology of the hydrosphere rather than of the atmos- 
phere. 

These frictional forces drag the moon back in its course about 
the earth, and accelerate the rotation of the earth forward. They 
lend to bring the lengths of the month and of the day ever closer 
to one another. Indeed, the day of the moon is the month, 
and the moon always presents nearly the same face to the earth. 
It has been suggested that this is the result of an ancient tidal 
evolution, when the moon contained some liquid or gas or 
plastic material which could give under the earth's attraction, 
and in so giving could dissipate large amounts of energy. This 


phenomenon of tidal evolution is not confined to the earth and 


the moon, but may be observed to some degree throughout all 
gravitating systems. 


In ages past it has seriously moditied the 
face of the solar system, though in anything like historic times, 
this modification is slight compared with the « rigid body » 
motion of the planets of the solar system, 


Thus even gravitational astronom 


y involves frictional proces- 
ses that run down. 


There is nota single science which conforms 
precisely to the strict Newtonian pattern. 


The biological sciences 
certainly hay 


e their full share of one-way phenomena. Birthis 
not the exact reverse of death, nor is anabolism, the building up 
of lissues, the exact reverse of catabolism, their breaking down. 
The division of cells does not follow a pattern symmetrical jn 
time, nor does the union of the germ-cells to form the fertilized 
ovum. The individual is an arrow pointed through time in one 
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way, and the race is equally directed from the past into the 
future. 

: The record of paleontclogy indicates a definite long-time trend, 
interrupted and complicated though it might be, from the simple 
to the complex. By the middle of thelastcentury this trend had 
become apparent to all scientists with an honestly open mind, 
and it is no accident that the problem of discovering its mecha- 
nisms was carried ahead through the same great step by two 
men working at about the same time : Charles Darwin and 
Alfred Wallace. This step was the realization that a mere for- 
tuitous variation of the individuals of a species might be carved 
into the form of a more or less one-directional or few-directional 
progress for each line by the varying degrees of viability of the 
several variations, either from the point of view of the indivi- 
dual or of the race. A mutant dog without legs will certainly 
starve, while a long thin lizard that has developed the mechanism 
of crawling on its ribs may have a better chance for survival if 
it has clean lines and is freed from the impeding projections of 
limbs. An aquatic animal, whether fish, lizard, or mammal, will 
swim better with a fusiform shape, powerful body muscles, and 
a posterior appendage which will catch the water ; and if it is 
dependent for its food on the pursuit ot swift prey, its chances 
of survival may depend on ils assuming this form. 

Darwinian evolution is thus hanism by which a more or 
less fortuitous variability is combined into a rather definite pat- 
tern. Darwin's principle still holds_ today, though we have a 
much potter knowledge of the mechanism on which it depends. 
The work of Mendel has given us a ‘far more precise and dis- 
continuous view of heredity than that held by Darwin, while the 
Nolion of mutation, from the time of de Vries on, has com- 
pletely altered our conception of the statistical basis of mutation. 
We have studied the fine anatomy of the chromosome, and have 
localized the gene 00 it. The list of modern geneticists is long 
and distinguished. Several of these, such as Haldane, have made 
the statistical study of Mendelianism an effective tool for the 


study of evolution. 
We have already $ 

Darwin, Charles Darw 

idea of the son with th 


a mec 


poken of the tidal evolution of Sir George 
in’s son. Neither the connection of the] 
at of the father, nor the choice of the 
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name, « evolution », is fortuitous. 


in the origin of species, we have 
which a fortuitous variability, 


In tidal evolution as well as 
a mechanism by means of 
that of the random motions of the 
waves in a tidal sea, and of the molecules of the water, is con- 
verted by a dynamical process into a pattern of development 
which reads in one direction. The theory of tidal evoln- 


tion is quite definitely an astronomical application of the elder 
Darwin. 


The third of the dynasty of D 


: ba oe : e 
arwins, Sir Charles, is one of th 
authorities on modern quantun 


: reiuee ’ abe 
n mechanics. This fact may b 

i i nee - Inva- 
fortuitous, but it nevertheless represents an even further invé 


; . o A cee — scession 
sion of Newtonian ideas by ideas of Stalislics. The successio 


Z ‘osressive 
of names Maxwell-Boltzmann-Gibbs represents a_ progressiv 


O slalistical mechanics : that is, 


@ concerning heat and temperature 
a Newtonian mech 


anics is applied to a 
al, not with 


a single dynamical system, 
dynamical systems; and in 
not all such systems but an 
About the year 1900 it became 
1g seriously wrong with ther- 
it concerned radiation. The 
to absorb radiations of high fre- 
aw ofsPlancle—="\than any existing 
heory had allowed. Planck gave a 
eory — which 


situation in which we de 
but to a statistical distribution of 
which our Conclusions concern 
overwhelming majority of them, 
apparent that there was somethir 
modynamics, particularly where 
ether showed much less power 
quency —as shown by the | 
mechanization of radiation t¢ 
quasi-atomic theory of radi 
accounted satisfactorily na, but which 
was at odds with the Ainder of physics ; and Nils 
Bohr followed this up witha Similarly ad hog theory of the atom. 
Thus Newton and Planck- €spectively the thesis 
and antithesis of a Hegeli The synthesis is the 
statistical theory discovered by Hei in 1925, in which the 
statistical Newtonian dynami 

tical theory very similar to that of 


fe collection of data 
to predict the future 
much lo say that not 
en the Newtonian phy- 
8e results of a statistical 


ufficient 
more than slatistically. It is thus Not too 


only the Newtonian. astronomy, but ey 
sics, has become a picture of the aver 
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situation, and hence an account of an evolutionary process. 

This transition from a Newtonian, reversible time to a Gibbs- 
ian, irreversible, time has had its philosophical echoes. Berg- 
son emphasized the difference between the reversible time of 
physics, in which nothing new happens, and the irreversible time 
ofsevolution and biology, in which -there is always something 
new. The realization that the Newtonian physics was not the 
proper frame for biology was perhaps the central point in the 
old controversy between vitalism and mechanism; although this 
was complicated by the desire to conserve in some form. or 
other at least the shadows of the soul and of God against the 
inroads of materialism. In the end, as we have seen, the vita- 
list proved too much. Instead of building a wall between the 
claims of life and those of physics, the wall has been erected to 
surround so wide a compass that both matter and life find them- 
selves inside it. It is true that the matter of the newer phy- 
sics is not the matter of Newton, but it is someting quite as 
remote from the anthropomorphising desires of the vitalists. 
The chance of the quantum-theorelician is not the ethical free- 
dom of the Augustinian, and Tyche is as relentless a mistress as 
Ananke. 

The thought of every age is reflected in its technique. The 
civil engineers of ancient days were land-surveyors, astronomers, 
and navigators ; those of the seventeenth and early eighteenth 
centuries were clockmakers and grinders of lenses. As in 
ancient times, the craflsmen made their tools in the image of 
the heavens. A watch is nothing but a pocket orrery, moving 
by necessity as do the celestial spheres; and if friction and the 
dissipation of energy play a role in it, they 
come, so that the resulting motion of the hands may be as perio- 


dic and regular as possible. The chief technical result of this 


engineering after the model of Hu. 
age of navigation, in which for the first time it was possible to 


compute longitudes with a respectable precision, and to convert 
the commerce of the great oceans from a thing of chance and 
adventure to a regular understood business. It is the enginee- 


ring of the mercantilists. 
To the merchant succeeded the manufacturer, and to the 


chronometer, the steam-engine. From the Newcomen engine 


yghens and Newton was the 


. 


are effects to be aba 
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almost to the present time, the central field of engineering Bas 
been the study of prime movers. Heat has been converted into 
usable energy of rotation and translation, and the physics of 
Newton has been supplemented by that of Rumford, Carnot, 
and Joule. Thermodynamics makes ils appearance, a science * 
in which time is eminently irreversible; and although the ear- 
lier stages of this science seem to represent a region of thought 
almost without contact with the Newtonian dynamics, the theory 
of the conservation of energy and the later statistical explana- 
tion of the Carnot Principle or second law of thermodynamics 
or principle of the degradation of energy — that principle which 
makes the maximum efficiency obtainable by a steam-engine 
depend on the working temperatures of the boiler and the con- 
denser — all these have fused thermodynamics and the Newlo- 


nian dynamics into the Statistical and the non-statistical aspecls 
of the same science, 


If the seventeenth and early eighteenth centuries are the ABE 
_ of clocks, and the later eighteenth and the nineteenth centuries 
constitute the age of steam-engines, the present time is the mG 
of communication and control. There isin electrical engineering 
asplit which is known in Germany as the split between the tech- 
nique ofstrong currents and the technique of weak currents, and 
Which we know as the distinction between power and commu- 
nication engineering. [tis this split which separates the age just 
past from that in which we are now living. Actually, commlu- 
nication engineering can deal with currents of any size whatever, 
and the movement of engines powerful enough to swing massive 
gun turrets; what distinguishes it from power engineering is that 
its’ main interest is not economy of energy but the accurate 
reproduction of a signal, 


This signal may be the tap of a key, 
to be reproduced as the t 


ap ofa telegraph receiver at the other 
end ; or it may be a sound transmit 


ted and received through 
the apparatus ofa telephone ; or it may be the turn of a ship's 
wheel, received as the angular position of the rudder. Thus 
communication engineering began with Gauss, Wheatstone, and 
the first telegraphers. It received its first reasonably scienti- 
fic treatment at the hands of Lord a BR Pa 5 


— A d Kelvin, after the failure of the 
first transatlantic cable in the middle of ast century ana 


the | 
from the eighties on, it was pe Viside who did the 


rhaps Heay. 
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most to bring it into a modern shape. ‘The discovery of radar 
and its use in the second world war, together with the exigencies 
of the control of anti-aircraft fire, have brought to the field a 
large number of well-trained mathematicians and physicists. 

. The wonders of the automatic computing machine belong to the 
same realm of ideas, which was certainly never so actively pur- 
sued in the past as itis at the present day. 

At every stage of technique since Daedalus or Hero of Alexan- 
dria, the ability of the artificer to producea working simulacrum 
of a living organism has always intrigued people, This desire 
to produce and to study automata has always been expressed 
in terms of the living technique of the age. Inthe days of magic, 
we have the bizarre and sinister concept of the Golem, that 
figure of clay into which the Rabbi of Prague breathed in life 
with the blasphemy of the Ineffable Name of God. In_the 

time of Newton, the automaton becomes the clockwork music 
box, with the little effigies pirouetting stiffly on top. In_the_ 
nineteenth century, the automaton is a glorified heat engine, 
burning some combustible fuel instead of the glycogen of the 
human muscles. Finally, the present aulomaton opens doors 
by means of photocells, or points guns to the place at which a 
radar beam picks up an airplane, or computes the solution of a 
differential equation. 


‘Neither the Greek nor the magical automaton lies along the 
main lines of the direction of development of the modern 
machine, nor do they seem lo have had much of an influence 
hought. It is far different with the 
This idea has played a very genuine 
history of modern philosophy, 


on serious philosophic t 
clockwork automaton. 
and important role in the early 


although we are rather prone to Ignore it. 
To begin with, Descartes considers the lower animals as au- 


tomata. This is done to avoid questioning the orthodox Chris- 
tian attitude that animals have no souls to be saved or damned. 
Just how these living automata function is something that Des- 
carles, so far as I know, never discusses. However, the 
important allied question of the mode of coupling of the human 
soul, both in sensation and in will, with its material environ- 
ment, is one which Descartes does discuss, although in a very 
unsatisfactory manner. He places this coupling in the one 
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median part of the brain known to him, the pineal gland. As 
to the nature of this coupling — whether or not it represents a 
direct action of mind on matter and of matter on mind — he is 
none too clear. He probably does regard it as a direct action 
in both ways ; but he attributes the validily of human experience 
in its action on the outside world to the goodness and honesty 
of God. 
The role attributed to God in this matter is unstable. Either 
God is entirely passive, in which case it is hard to see how 
Descartes’explanation really explains anything, or Heis an active 
participant, in which case it is hard to see how the guarantee 
given by His honesty can be anything but an active participation 
in the act of sensation. Thus the causal chainof material pheno- 
mena is paralleled by a causal chain starting with the act of 
God, by which He produces in us the experiences corresponding 
toa given material situation. Once thisis assumed, it is entirely 
nalural to attribute the correspondence between our will and 
the effects it seems to produce in the external world to a similar 
divine intervention. This is the path followed by the Occa- 
Sionalists, Geulincx and Malebranche. In Spinoza, who is in 
many ways the continuator of this school, the doctrine of Occa- 
sionalism assumes the more reasonable form of asserting that 


spondence between mind and matter is that of two 
self-contained attributes of God ; but Spinoza is not dynamically 
minded, and gives little or no 


: attention to the mechanism of 
this correspondence. 


This is the situation from which Leibniz starts, but Leibniz 
is as dynamically minded as Spinoza is geometrically minded 
First, he replaces the pair of corresponding elements, mind 
and matter, by a Continuum of corresponding Hitmen Nine 
monads. While these are Conceived after the attenn ofthe: 
soul, they include many instances which do not te totheldearee 
of selfconsciousness of full souls, and which form part Bhat 
world which Descartes would have attributed to m as » Each 
of them lives in its own closed Universe, with A . a Sent 
chain from the creation or from minus infinity at rfect ¢ 

indefinitely remote future ; lyin time to the 


but closeq tho 
ugl 
correspond one to the other through the Pre-established neem 


of God. Leibniz compares them to clocks, which have so been 
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wound up as to keep time together from the creation for all 
eternity. Unlike humanly made clocks, they do not drift 
into asynchronism ; but this is due to the miraculously perfect 
workmanship of the Creator. 

Thus Leibniz considers a world of automata which, as is natu- 
ral in a disciple of Huyghens, he constructs after the model of 
clockwork. Though the monads reflect one another, this reflec- 
tion does not consist in a transfer of the causal chain from one 
to another. They are actually as self-contained as, or rather 
more self-contained than, the passively dancing figures on 
top of a music-box. They have no real influence on the out- 
side world, nor are they effectively influenced by it. As he 
says, they have no windows. The apparent organization of 
the world we see is something between a figment and a miracle. 
onian solar system writ small. 

In the nineteenth century, the automata which are humanly 
constructed, and those other natural automata, the animals and 
terialist, are studied from a very different aspect. 
e degradation of energy are the ruling 
living organism is above all a heat 
engine, burning glucose or glycogen or starch, fats, and proteins, 
inlo carbon dioxide, water, and urea. Itis the metabolic balance 
which is the center of attention; and if the low working tempe- 
tatures of animal muscle attract attention as opposed to the 
liigh working temperatures of a heat engine of similar efficiency, 
this factis pushed into a corner and glibly explained by a contrast 
between the chemical energy of the living organism and the 
thermalenergy of the heat engine. All the fundamental notions 
are those associated with energy, and the chief of these is that of 
potential. ‘The “engineering ‘of the body is a branch of power 
engineering © Even today, this is the predominating point of 
view of the more classically minded, conservative physiologists ; 
aiid the whole trend of thought of such biophysicists as Rashevsky. 

nd his school bears witness to its continued potency. 

"Today, we are coming to realize that the body is very far from 
a conservative system, and that its component parts work in an 
énvironment where the available power is much less limited 
{han we have taken it to be. The electronic ‘tube has shown us that 


The monad is a Newt 


plants of the mate 
The conservation and the 
principles of the day. The 


asystem with an outside source of energy, almost all of which 


v 


Metabolism, but also 
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is wasted, may be a very effective agency for performing desired 
operations, especially if itis worked at a low energy level. We 
are beginning to see that such important elements as the neurons, 
the atoms of the nervous complex of our body, do their work 
under much the same conditions as vacuum tubes, with their 
relatively small power supplied from outside by the circulation ; 
and that the bookkeeping which is most essential to describe 
their function is not one of energy. In short, the newer study 
of automata, whether in the metal or in the flesh, is a branch of 
communication engineering, and its cardinal notions are those 
of message, amount of disturbance or « noise » — a term taken_ 
over from the telephone engineer — quantity of information, 
coding lechnique, and so on. 

In such a theory, we deal with automata effectively coupled 
to the external world, not merely by their energy flow, their 
by a flow of impressions, of incoming 
messages, and of the actions of outgoing messages. The organs 
by which impressions are received are the equivalents of the 
human and animal Sense-organs. They comprise photo-electric 
cells and other receptors for light ; radar systems, receiving their 
own short Herizian Waves ; hydrogen-ion-potential recorders, 
which may be said to taste ; thermometers, pressure-gauges of 
various sorts, microphones and so on. The effectors may be 


electrical motors or solenoids or heating-coils or other instru- 
ments of very diverse sorts. Between the r 


and the effector stands an intermediate set of elements, whose 
function is to nscotabine the incoming impr 

form as to produce a desired type of response in the effectors. 
The information fed into this central control System will very 


often contain information concerning the functioning of the 
effectors themselves. These corr 


: : €spond among other things to 
the kinaesthetic organs and other Proprioceptors of the human 
system, for we too have organs 


: which record the positon ofa 
joint or the rate of contr 


joi action of q muscle, etc, Moreover, the 
information received by the automaton 


need not be used at 
once, but may he delayed or Stored so as to become ayailable at 
some future time. This is the analogue of memory. Finally 
as long as the automaton is running, its y ay 


ery rules of operation 
are susceptible to some change on the basis of the data Which 


eceptor or sense-organ 


essions into such 
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have passed through its receptors in the past, and this is not 
unlike the process of learning. 

The machines of which we are now speaking are not the 
dream of the sensationalist, nor the hope of some future time. 
They already exist as thermostats, automatic gyro-compass ship- 
steering systems, self-propelled missiles — especially such as seek 
their target — anti-aircraft fire-control systems, aulomatically 
controlled oil-cracking stills, ultra-rapid computing machines, 
and the like. They had begun to be used long before the war 
— indeed, the very old steam-engine governor belongs among 
them — but the great mechanization of the second world war 
brought them into their own, and the need of handling the 
extremely dangerous energy of the atom will probably bring 
them to a still higher point of development. Scarcely a month 
passes but a new book appears on these so-called control 
mechanisms or servo-mechanisms, and the present age is as 
truly the age of servo-mechanisms as the nineteenth century was 
the age of the steam engine or the eighteenth century the age of 
the clock. 

To sum up: the many automata of the present age are coupled 
to the outside world both for the reception of impressions and 
for the performance of actions. They conlain sense-organs, 
effectors, and the equivalent of a nervous system to integrate 
the transfer of information from the one to the other. They lend 
themselves very well to description in physiological terms. It is 
scarcely a miracle that they can be subsumed under one theory 
with the mechanisms of physiology. 

The relation of these mechanisms to time demands careful 
study. It is clear of course that the relation input-output is a 
consecutive one in time, and involves a definite past-future 
order. What is perhaps not so clear is that the theory of the 
sensitive automata is a statistical one. We are scarcely ever 
interested in the performance of a communication-engineering 
machine for a single input. To function adequately it must give 
a satisfactory performance for a whole class of inpuls, and this 
means a statislically satisfactory performance for the class of 
input which it is statistically expected to receive. Thus its 
theory belongs to the Gibbsian statistical mechanics rather than 
fo the classical Newtonian mechanics. We shall study this in 
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much more detail in the chapter devoted to the theory of com- 
munication. 

Thus the modern automaton exists in the same sort of Bergso- 
nian time as the living organism; and hence there is no reason 
in Bergson’s considerations why the essential mode of functioning 
of the living organism should not be the same as that of the 
automaton of this type. Vitalism has won to the extent that even 
mechanisms correspond to the time-structure of vitalism ; but 
as we have said this victory is a complete defeat, for from every 
point of view which has the slightest relation to morality or 
religion, the new mechanics is fully as mechanistic as the old. 
Whether we should call the new point of view materialistic is 
largely a question of words : the ascendancy of matter charac- 
terizes a phase of nineteenth-century physics far more than the 
present age, and « materialism » has come to be but little more 
than a loose synonym for « mechanism». In fact, the whole 
mechanist-vitalist controversy has been relegated to the limbo 
of badly posed questions. 


oe ee 
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CHAPTER II 


GROUPS AND STATISTICAL MECHANICS 


At about the beginning of the present century, two scientists, 
one in the United States and one in France, were working along 
lines which would have seemed to each of them entirely unre- 
lated, if either had had the remotest idea of the existence of 
the other. In New Haven, Willard Gibbs was developing his 
new point of view in statistical mechanics. In Paris, Henri 
Lebesgue was rivalling the fame of his master Emile Borel by 
the discovery of arevised and more powerful theory of integration 
for usein the study of trig nomeltric series. The two discoverers 
were alike in this, that each was a man of the study rather than 
of the laboratory, but from this point on, their whole attitudes 
to science were diametrically opposite. 

Gibbs, mathematician though he was, always regarded mathe- 
matics as ancillary to physics. Lebesgue was an analyst of the 
purest type, an able exponent of the extremely exacting modern 
standards of mathematical rigor, and a writer whose works as 
far as I know do not contain one single example of a problem 
or a method originating directly from physics. Nevertheless, 
the work of these two men forms a single whole in which the 


questions asked by Gibbs find their answers, not in his own 


work, but in the work of Lebesgue. 

The key idea of Gibbs is this : in Newton’s dynamics, in its 
cerned with an individual system, 
and momenta, undergoing changes 
according to a certain system of forces under the Newtonian 
laws which link force and acceleration. In the vast majority 
of practical cases, however, We are far from knowing all the 
initial velocities and momenta. If we assume a certain initial 
distribution of the incompletely known positions and momenta 


original form, we are con 
with given initial velocities 


58 


CYBERNETICS 


of the system, this will determi 
way the distribution of the m 
future time. It will then be po 
these distributions, and some of 
assertions that the future sys 
with probability 
bability zero. 
Probabilities one and zero 


ne in a completely Newtonian 
omenta and positions for any 
ssible to make statements about 
these will have the character of 
tem will have certain characteristics 
one, or certain other characteristics with pro- 


are notions which include complete 
certainty and complete impossibility, but include much moré 
as well. If I shoot at a target with a bullet of the dimensions 
of a point, the chance that I hit any specific point on the target 
will generally be zero, although it is not impossible that I hit it; 
and indeed, in each specific case | must actually hit some 
Specific point, which is an event of probability zero. Thus an 
event of Probability one, that of my hitting some point, may 


be made up of an assemblage of instances of probability 
zero. 


Nevertheless, one of the 
nique of the Gibbsi 
implicitly 
lution of 


Processes which is used in the tech- 
an stalistical mechanics, although it is used 
»and Gibbs is nowhere clearly aware of it, is the reso- 
# complex contingency into an infinite sequence of 


more special contingencies — q first, a second, a third, and so 
on — each of which has a known probability ; 


of the probability of this larger contingency 
Probabilities of the more special conti 
infinite sequence, Thus we cannot 

conceivable cases, to get a probabilit 
the sum of any number 
them if there is a first, a second 
forming a sequence of co 
definite position given by a Positive integer, 

The distinction between 


and the expression 
as the sum of the 
ngencies, which form an 
Sum probabilities in all 
y of the total event — for 


> a third Member, and so on, 
ntingencies in Ww 


rather subtle 
nstances, and 

Gibbs, although a VEryi (p r ian, was never a 
very subtle one. Isit possi infinite, and yet 
: another infinite class, 
Cmtlegers? This Problem was solved 
by Georg Cantor, and the 
er all the distinct decimal 
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fractions, terminating or non-lerminaling, lying between zero 
and one, itis known that they cannot be arranged in one, two, 
three order - although, strangely enough, all the terminating 
decimal fractions can be so arranged. Thus the distinction 
demanded by the Gibbs statistical mechanics is not on the face 
of it an impossible one. The service of Lebesgue to the Gibbs 
theory is to show that the implicit requirements of statistical 
mechanics concerning contingencies of probability zero and 
the addition of the probabilities of contingencies can actually 
be met, and that the Gibbsian theory does not involve can- 
tradictions. a 
Lebesgue’s work, however, was not directly based on the 
needs’ of statistical mechanics, but on what looks like a very 
different theory, the theory of trigonometric series. This goes 
back to the eighteenth-century physics of waves and vibrations, 
and to the then moot question of the generality of the sets 
of motions of a linear system which can be synthesised out of 
the simple vibrations of the system — out of those vibrations, 
in other words, for which the passing of time simply multiplies 
the deviations of the system from equilibrium by a quantity, 
positive or negalive, dependent on the lime alone, and not on 
position. Thus a single function is expressed as the sum of a 
series. In these series, the coefficients are expressed as averages 
of the product of the function to be represented, multiplied by 
a given weighting function. The whole theory depends on the 
properties of the average ofaseries, in terms of the average of an 


individual term. Notice that the average of a quantity which 
val from zero to A, and zero from A to 1, 
as the probability that the random 


point should lie in the inter ‘al from 0 to A, if it is known to lie 
between ( and 1. In other words, the theory needed for the 
average of a series is very close to the theory needed for an ade- 
quate discussion of probabilities compounded from an infinite 
sequence of cases. ‘This is the reason why Lebesgue, in solving 


his own problem, had also solved that of me. 
The particular distributions discussed by Gibbs have themselves 


a dynamical interpretation. If we consider a cerlain very 


general sort of conservative dynamical system, with N degrees 


of freedom, we find that its position and_ velocity coordinates 


is one over an inter 
is A, and may be regarded 
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may be reduced to a special set of 2 N coordinates, N of which 
are called the generalized position coordinates and N the gene- 
ralized momenta. These determine a 2N-dimensional space 
defining a 2N-dimensional volume ; and if we take any region 
of this space, and let the points flow with the course of time, 
which changes every set of 2N coordinates into a new set 
depending on the elapsed time, the continual change of the 
boundary of the region does not change its 2N-dimensional 
volume. In general, for sets not so simply defined as these regions; 
the notion of volume generates a system of measure of the type 
of Lebesgue. {n this system of measure, andin the conservative 
dynamical systems which are transformed in such a way as to 
keep this measure constant, there is one other numerically 
valued entity which also remains constant : the energy. If all 
the bodies in the system act only on one another, and there are 
“no forces attached to fixed positions and fixed orientations 2 
Space, there are two other expressions which also remain 
constant. Both of these are vectors ; the momentum, and the 
moment of momentum of the system as a whole. They are not 
difficult to eliminate, so that the sy 
with fewer degrees of freedom. 

In highly specialized systems, there may be other quantities, 
not determined by the energy, the momentum, and the moment of 
momentum, which are unchanged as the system develops. 
However, it is known that systems in which another invariant 
quantity exists, dependent ontheinitial coordinates and momenta 
of a dynamical system, and regular enough to be subject to the 
system of integration based on Lebesgue measure, are very rare 


indeed ina qtlile precise sense. In systems without other inva- 
riant quantities, we can fix the 


stem is replaced by a system 


es, the measure determined by 


rdinates wil] itself determine a 
sort of sub-measure, just as measure in Space will determine 


area on a two-dimensional surface out of a family of two-dimen- 
sional surfaces. For example if our family is that of concentric 
spheres, then the volume between two Concentric spheres close 


1. Oxtoby J. Cand Ulam, « Measure-Preserving Homeomorphisms nd Metri— 
eal transivity ». Ann. of Math. (2), 42, 874-999 (1941), ie 2 
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together, when normalized by taking as one the total volume of 
the region between the two spheres, will give in the limit a mea- 
sure of area on the surface of a sphere. 

Letus then takethis new measure on aregion in phase space for 
which energy, total momentum, and total moment of momentum 
are determined, and let us suppose that there are no other 
measurable inyariant quantities in the system. Let the total 
measure of this restricted region be constant, or as we can make 
itby a change in scale, one. As our measure hasbeen obtained 
from a measure invariant in time, ina way invariant in time, 
it will itself be invariant. We shall call this measure phase 
measure, and averages taken with respect to it phase averages. 

However, any quantily varying in lime may also have a lime 
average. If, for example, f() depends on ¢, ils time average for 
the past will be : 


2 1 0 
(2.01) ee Fa J fil) dt; 
oe 
and its time average for the future : 
A 1 tT 
(2,02) eal [Ko at. 
) 


In Gibbs’ statistical mechanics, both time averages and space 
averages occur. It was a prilliant idea of Gibbs to try to show 
that these two types of average were, in some sense,. the same. 
In thenotion that these two types of average were related, Gibbs 
was perfectly right ; and in the method by which he tried to 
show this relation, he was utterly and hopelessly wrong. For 
this he was scarcely to blame. Even at the time of his death, 


the fame of the Lebesgue integral had just begun to penetrate to 


America. For another fifteen years, it was a museum curiosity, 


only useful to show to young mathematicians the needs and 
possibilities ofrigor. A mathematician as distinguished as W.F. 
Osgood 1, would haye nothing to do with it till his dying day. 
It was not until about 1930 when a group of mathematicians — 
Koopman, von Neumann, Birkhoff *, — finally established the 


1. Nevertheless some of Osgood’s early work represented an important step 
in the direction of the Lebesgue Integral. 
2. Hopf, E.. « Ergodentheorie ». Erg. d. Math. (1936). 
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— ome r 1 
proper foundations ofthe Gibbs statistical mechanics. Weshal 


: : » of ergodic 
later see what these foundations were, in the study of ergod 
theory. — 


Gibbs himself thought that in a system from which all the 
invariants had been removed as extra coordinates, almost all 
paths of points in phase-space passed through all cuordinites 
insuchaspace. This hypothesis he called the ergodic ccguene 
from the Greek words 2 yor, « Work », and 4202, « path ». ea. 
in the first place, as Plancherel, and others have shown, nee 
is.no significant case where that hypothesis is true. No sal 
rentiable path can cover an area in the plane, even if it 1s _ 
infinite length. The followers of Gibbs, including at the ne 
perhaps Gibbs himself, saw this in a vague way, and wane 
this hypothesis by the quast-ergodic hypothesis, which merely 
asserts that in the course of time, a system generally passes inde- 
finitely near to every point in the region of phase-space deter- 
mined by the known invariants. There is no logical difficulty 
as to the truth of this : it is merely quite inadequate for the 
conclusions which Gibbs bases on it. It says nothing about the 


: 3 : . ‘ e f 
relative time which the system spends in the neighborhood 0 
each point. . 


Besides the notions of average 2 
over a Universe of 


. average 
tnd of measure — the averag 
a function one over 


a set to be measured and 
zero elsewhere — which wer 


© most urgently needed to make 
sense out of Gibbs’ theory, in order to appreciate the real signifi- 
cance of ergodic theory, we need a more precise analysis of the 
notion of invariant, as well as the notion of transformation group. 
These notions were certainly familiar to Gibbs, as his study of 
Sector analysi shows. Nevertheless, it is possible to maintain 
that he did not assess them at their full Philosophical value. 
Like his contemporary Heaviside, Gibbs is one of the scientists 
whose physico-mathematical acumen often outstrips his logic, 
and who is generally right, while he is often unable to explain 
why and how he is right. 

For the existence of any science, it 
exist phenomena which do Not stan 
by a succession of miracles perfo 
subject to sudden whims, we sho 
new catastrophe in a state of per 


is necessary that there 
d isolated, In a world ruled 
rmed by an irrational God 
uld be forced to await each 
plexed Passiveness. We have 
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a picture of such a world in the croquet game in Alice in Won- 
derland ; where the mallets are flamingos; the balls, hedgehogs, 
which quietly unroll and go about their own business; the hoops, 
playing-card soldiers, likewise subject to locomotor initiative 
of their own;and the rules are the decrees of the testy, unpre- 
dictable Queen of Hearts. 

The essence of an effective rule for a game or a useful law of 
physics is that it be stateable in advance, and that it apply to 
more than one case. Ideally, it should represent a property of 
the system discussed which remains the same under the flux-of 
particular circumstances. In the simplest case, itis a property 
which is invariant to a set of transformations to which the system 
is subject. We are thus led to the notions of dransformation, 
transformation-group, and invariant. 

A transformation of a system is some alteration in which each 
element goes into another. The modification ef the solar system 
which occurs in the transition between lime ¢, and time @, is a 
transformation of the sets of coordinates of the planets. The 
similar change in their coordinates when we move their origin, 
or subject our geometric axes to a rotation, is a transformation. 
The change in scale which occurs when we examine a preparation 
under the magnifying action of a microscope is likewise a trans- 
formation. . 

The result of following a transformation A by a transforma- 
tion B is another transformation, known as the product or resul- 
tant BA. Note that in general it depends on the order of A 
and B. Thus if A is the transformation which takes the coordi- 


natex into the coordinate y,andyinto —2x,while zis unchanged ; 


while B tales a into z, z into — 2, and y is unchanged ; 
then BA will take x into y, y into — = and z into —- x; while AB 


will take « into z, y into — 2, and z into —y. If AB and BA 
B are permutable. 


are the same, we shall say that A and 
but not always, the transformation A will not 


Sometimes, 
stem into an element, but will 


only carry every element of the sy 
have the property that every element is the result of transfor- 
mingan element. In this case, there is a unique transformation 
A-1, such that both AA and A“A are that very special transfor- 
mation which we call I, the identity transformation, which trans- 
forms every element into itself. In this case we call A~ the 
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inverse of A. It is clear that A is the inverse of A 1, that Lis its 
Own inverse, and that the inverse of AB is B-1A°2. P ‘ 

There exist certain sets of transformations where every {rales 
formation belonging to the set has an inverse, likewise belonging: 
to the set ; and where the resultant of any two transformations 
belonging to the set itself belongs to ‘the set. These sets are 
known as transformation-groups. The set of all translations 
along a line, or ina plane, or in a three-dimensional space, 1S 
a transformation group;and even more, it is a transformation 
group of the special sort known as Abelian, where any two Mans 
formations of the group are permutable. The set of rotations 
about a point, and of all motions of a rigid body in space, are 
non-Abelian groups. 

Let us suppose that we have some quantity attached to al! the 
elements transformed by a transformation group. If this quan- 
tity is unchanged when each element is changed by the same 
transformation of the group, whatever that transformation may 
be, it is called an invariant of the group. There are many sorts 
of such group invariants, of which two 
tant for our purposes. 

The first are the so-called line 
transformed by an 
sent by a, 


are especially impor- 


ar invariants. Let the elements 
Abelian group be the terms which we repre- 
and let f(z) be a complex-valued function of these 
elements, with certain appropriate properties of continuity oF 
integrability. Then if Tx Slands for the element resulting from 


x under the transformation T, and if {(x) isa function of abso- 
lute value 1, such that: 


(2.03) [(Tx) = (T) fe), 
where o(T) is a number of absolute y. 
T, we shall say that f(x) is a charact 
invariant of the group in a slightly Seneralized sense. If /(2) 
and g(x) are group characters, clearly I(x) g(x) is one also, as 
is [f(z)| +. If we can represent any function A(x) defined over 
the group as alinear combination of the characters of the group, 
in some such form as. = 


alue 1 depending only on 
er of the group. It is an 


(2.04) h@) = A, fitz), 
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where f;/2) is a character of the group, and @(T) bears the same 
relation to f(x) that «(T) does to f(x) in (2.03), then 


(2.05) A(Tx) = Sy Ax ex(T) fide). 


Thus if we can develop h(x) in terms of a set of group charac- 
ters, we can develop A(@x) for all T in terms of the characters. 

We have seen that the characters of a group generate other 
characters under multiplication and inversion, and it may simi- 
larly be seen that the constant 1 is a character. Multiplication 
by a group character thus generates a transformation group of 
the group characters themselves, which is known as the cha- 
racter group of the original group. 

If the original group is the translation group on the infinite 
line, so that the operator T changes x into a+ T, (2.03) be- 
comes : 

(2.06) fix +1) = (7) fa), 

which is satisfied if /(v) =e", a(L)=e?T, The characters will be 
the functions e, and the character group will be the group of 
translations changing 2 intoa++, thus having the same structure 
as the original group. ‘This will not be the case when the original 
group consists of the rotations about a circle. In this case, the 
operator T changes X into a number between 0 and 2z, differing 
from a + T by an integral multiple of 2x, and while (2.06) will 
still hold, we have the extra condition that 


(2.07) a(‘T + 2n) = o(T). 
If now we put /(x) =e as before, we shall obtain : 


(2.08) enh = 1, 
which means that 4 must be a real integer, positive, negative, or 
zero. Thecharacter group thus corresponds to the translations 
of the real integers. If on the other hand the original group is 
that of the translations of the integers, x and T in (2.05) are con- 
fined to integer values, and ee involves only the number het- 
ween 0) and 2x which differs from » by an integral multiple of 2z. 
Thus the character group is essentially the group of rotations 


about acircle. . 
In any character group, for a given character f, the values of 
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«(T) are distributed in such a way that the distribution is not 
altered when they are all multiplied by «(S), for any element S 
in the group. That is, if there is any reasonable basis of taking 
an average of these values which is not affected by the trans- 
formation of the group by the multiplication of each transforma- 
tion bya fixed one ofits transformation, either «(T) is always 1, 
or this average is invariant when multiplied by some number 
not 1, and must be 0. From this it may be concluded that the 
average of the product of any character by its conjugate (which 
willalso be a character) will have the value 1, and that the ave- 
rage of the product of any character by the conjugate of another 
character will have the value 0. In other words, if we can 
express h(a) as in (2.04), we shall have: 


(2.09) Ax= Average (h(x) f(a) ). 
In the case of the group of rotations ona circle, this gives us 
directly that if 
(2.10) (x) = > An einz, 


then : 


(2.11) Ge — J fe) mine. dap = 
an 
0 


and the result for translations along the infinite line is closely 
velated to the fact that if in an appropriate sense, 


(2.12) [(x) = [a ede dh, 


then in a certain sense 
i 7s 
2.13 j= — r) Eide dp 
(2.13) a(a) rae flax) D2 dy, 


These results have been Stated here very roughly, and without 
a clear statement of their conditions of 


‘validity. For more pre- 
cise statements of the theory, the reader should consult the 
following references }, 


Besides the theory of the linear invariants of a group, there 


1. Wiener, The Fourier Inte 


gral and Certain of 
Univ, Press (1932). 


ils Applications. Cambridge 
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is also the general theory of its metrical invariants. These are 
the systems of Lebesgue measure which do not undergo any 
change when the objects transformed by the group*are permu- 
ted by the operators of the group. In this connection, we should 
cite the interesting theory of group measure, due to Haar’. 
As we have seen every group itself is a collection of objects 
which are permuted by being multiplied by the operations of 
the group itself. As such, it may have an invariant measure. 
Haar has proved that a certain rather wide class of groups does 
possess a uniquely determined invariant measure, definable. in 
terms of the structure of the group itself. ' 

The most important application of the theory of the metrical 
invariants ofa group of transformations is to show the justification 
of that interchangeability of phase averages and time averages 
which, as we have already seen, Gibbs tried in vain to establish. 
The basis on which this has been accomplished is known as the 
ergodic theory. 

The ordinary ergodic theorems start with an ensemble E, 
which we can take to be of measure 1, transformed into itself 
by a measure-preserving transformation T, or by a group of 
measure-preserving transformations T?, where — «<< cand 
where 

(2.14) 

Ergodic theory concerns itself with complex-valued functions 


f(x) of the elements x of E. In all cases, f(z) is taken to be 
and if we are concerned with a continuous 
rx) is taken to be measurable in x 


Tete Te 


measurable in 2, 
group of transformations, f(T 


and simultaneously. 
In the mean ergodic theorem of Koopman and von Neumann, 


f(x) is taken to be of class L?; that is, 


(2.15) f {(x) |? dxr< & 


The theorem then asserts that 


1, fete 


1. Haar, H., « Der Maassbegriff in der Theorie der Kontinuierlichen Gruppen ». 
Ann. of Math. (2) 34, pp. 147-169 (1933). 
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or 


1 A 
(2.17) fi) =— [(Trx) do, 
toy f 


as the case ‘may be, converges in the mean to a limit /*(a’) as 
N-— o or A — o respectively, in the sense that : 


CONS ea pi | f#(x) — fa(x) |P de = 0 
eeroy ett 7. | f(a) — f(a) |? de = 0. 


In the « almost everywhere » ergodic theorem of Birkhoff, 
f(x) is taken to be of class L; which means that 


(2.20) af | fx) | dx < o. 


The functions fs(a) and f,(a) are defined asin (2.16) and (2.17). 
The theorem then states thal, except for a set cf values of © 
measure 0, 


(2.21) P(e) = yg fale) 
and 

(2.22) fray = 8" AG) 
exist. 


A very interesting case is the so-called 


ergodic or metrically 
transitive one 


» in which the transformation T, or the set of 
transformations T>, leaves invariant no set of points 2 which 
has a measure other than 1or 0. In such a case, the set of values 


(for either ergodic theorem) for which {*(x) takes on a certain 
range of value is almost always either 1 or 0. 


unless /*() is almost always constant. 
then assumes almost always, is 


(2.23) We) dz. 
ip 


That is, in the Koopman theorem, w 


This is impossible 
The value which /*(2) 


e have 
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z N 
(2.24) ik tks fra) = ffx) dx ; 


and in the Birkhoff theorem, we have 


— li 1 NE as 
(2.25) Nos hes 3 Arm = fn a 
ne 5; 


except for a set of values of a of zero measure or probability 0. 
Similar results hold in thecontinuous case. This is anadequate 
justification for Gibbs’ interchange of phase averages and time 
averages. 

Where the transformation T or the transformation group T* is 
not ergodic, von Neumann has shown under very general condi- 
tions that they can be reduced to ergodic components. Thatis, 
exceptfora set of values of 2 of zero measure, Ecan be separated 
into a finite or denumerable set of classes E, and a continuum 
of classes E(y), such thata measure is established on each E, and 
E(y), which is invariant under T or T?. These transformations are 
all ergodic; and if S(y) is the intersection of S with E(y) andS, with 
E,, then 


v 


(2.26) measure (S) = { measure (S(y) ) dy +3, measure (Sn). 
E Ey En 


In other words, the whole theory of measure-preserving transfor- 
mationscan be reduced to the theory of ergodic transformations. 

The whole of ergodic theory, let us remark in passing, may 
be applied to groups of transformations more general than 
those isomorphic with the translation group on the line. In 
particular, it may be applied to the translation group in 4 dimen- 
sions. The case of three dimensions is physically important. 
The spatial analogue of temporal equilibrium is spatial homoge- 
neity, and such theories as that of the homogeneous gas, liquid, or 
solid depend on the application of three-dimensional ergodic 
theory. Incidentally, a non-ergodic group of translation transfor- 
mations in three dimensions appears as the set of translations ofa 
mixture of distinct states, such that one or another exists ata given 


time, not a mixture of both. 
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One of the cardinal notions of statistical mechanics, which 
also receives an application in the classical thermodynamics, is 
that of entropy. It is primarily a property of regions in phase- 
space, and expresses the logarithm of their probability measure. 
For example, let us consider the dynamics of n particles in a 
bottle, dividedinto two parts, Aand B. If m particles are in A, 
and n-m in B, we have characterized a region in phase-space, 
and it will have acertain probability measure. The logarithm 
is the entropy of the distribution : m particles in A, n-m in B. 
The system will spend most of its time in a state near that of 
greatest entropy, in the sense that for most of the time, nearly 
particles will be in A, nearly n-m, in B, where the probability of 
the combination m, in A, n-m in B is amaximum: For systems 
with a large number of particles, and states within the limits of 
practical discrimination, this means that if we take a state of 
other than maximum entropy, and observe what happens to it, 
the entropy almost always increases. 

In the ordinary thermodynamic problems of the heat engine, 
we are dealing with conditions in which we have a rough thermal 
equilibriumin large regions like an engine cylinder. The states 
for which we study the entropy are states involving maximum 
entropy for a given temperature and volume, for a small number 
of regions of the given volumes and at the given temperature 
assumed. Even the more refined discussions of thermal engines, 
particularly of thermal engines like the turbine, in which a gas is 
expanding in a more complicated manner than ina cylinder, do 
not change these conditions too radically. We may still talk of 
local temperatures, with a very fair approximation even though 
po temperature is precisely determined except ina state of equi- 
librium and by methods involving this equilibrium. However, in 
living matter, we lose much of even this roughhomogeneity. The 
structure of a protein tissue as shown by the electron microscope 
has an enormous definiteness and fineness of texture, and its 
physiology is certainly of a corresponding fineness of texture. 
This fineness is far greater-than that of the space-and-time scale 
of an ordinary thermometer, and so the temperatures read by 
ordinary thermometers in living tissues are gross averages, and 


not the true temperatures of thermodynamics. Gibbsian statis- 


tical mechanics may well be a fairly adequate model of what 
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happens in the body; the picture suggested by the ordinary 
heat engine certainly is not. The thermal efficiency of muscle 
action means nexl to nothing, and certainly does not mean 
what it appears to mean. 

A very important idea in statistical mechanics is that of the 
Maxwell demon. Let us suppose a gas in which the particles 
are moving around with the distribution of velocities in statisti- 
cal equilibrium fora given temperature. Fora perfect gas, this is 
the Maxwell distribution. Let this gas be contained in a rigid 
container with a wall across it, containing an opening spanaed 
by a small gate, operated by a gatekeeper, either an anthropo- 
morphic demon or a minute mechanism. When a particle of 
more than average velocity approaches the gate from compart- 
ment A ora particle of less than average velocity approaches the 
gate from compartment B, the gatekeeper opens the gate, and 
the particle passes through; but when a particle of less than ave- 
rage velocity approaches from compartment A or a particle of 
greater than average velocity approaches from compartment B, 
the gate is closed: In this way, the concentration of particles 
of high velocity is increased in compartment B and is decreased 
in compartment A. This produces an apparent decrease in 
entropy; so that if the two compartments are now connected by 
a heat engine, we seem to obtain a perpetual-motion machine 
of the second kind. 

It is simpler to repel the question posed by the Maxwell 
demon than to answer it. Nothing is easier than to deny the 
Possibility of such beings or structures. We shall actually find 
that Maxwell demons in the strictest sense cannot exist in a 
system in equilibrium, but if we accept this from the beginning, 
and do not try to demonstrate it, we shall miss an admirable 
Opportunity to learn something about entropy and about 
possible physical, chemical, and biological systems. 

For a Maxwell demon to act, it must receive information from 
approaching particles, concerning their velocity and point of 
impact on the wall. Whether these impulses involve a transfer 
of energy or not, they must involve a coupling of the demon 
and the gas. Now, the law of the increase of entropy applies 
to a completely isolated system, but does not apply to a non- 
isolated part of such a system. Accordingly, the only entropy 
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which concerns us is that*of the system gas-demon, and aah 
that of the gas alone. The gas entropy is merely ‘one oe m 
in the total entropy of the larger system. Can we find ye 
involving the demon as well which contribute to this tola 
opy ? 
Be ercstinicly wecan. The demon can only act on informa- 
tion received, and this information, as we shall see oad the next 
chapter, represents a negative entropy. The fetornratien: mninst 
be carried by some physical process, say some form of radiation. 
It may very well be that this information is carried at a very low 
energy level, and that the transfer of energy between particle 
and demon is for a considerable time far less significant than 
the transfer of information. However, under the quantum 
mechanics, it is impossible to obtain any information giving the 
position or the momentum of a particle, much less the tye 
together, without a positive effect on the energy of the particle 
examined, exceeding a minimum dependenton the frequency of 
the light used for examination. Thus all coupling is strictly 
Coupling involving energy ; anda system in statistical equilibrium 
is in equilibrium both in matters concerning entropy and those 
concerning energy. In the long run, the Maxwell demon is itself 
subject to a random motion corresponding to the temperature of 
its environment, and as Leibniz says of some of his monads, it 
receives a large number of small impressions, until it falls into 
« a certain vertigo », and is incapable of clear perceptions. 
In fact, it ceases to act as a Maxwell demon. 

Nevertheless, there may be 
before the demon is decond 
prolonged that we may spea 
as metastable. 


a quite appreciable interval of time 
itioned, and this time may be so 
k of the active phase of the demon 
There is no reason to suppose that metastable 
demons do not in fact exist ; indeed, it may well be that enzymes 
are metastable Maxwell demons, decreasing entropy, perhaps 
not by the separation between fast and slow particles, but by 
some other equivalent process. We may well regard living 


organisms, such as Man himself, in this light. Certainly the 


enzyme and the living organism are alike metastable: the stable 
state of an enzyme is to be deconditioned, and the stable state 
ofa living organism is to be dead. All c 


atalysts are ultimately 
poisoned : they 


change rates of reaction, but not true equili- 
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brium. Nevertheless, catalysts and Man alike have sufficiently 
definite states of melastability to deserve the recognition of these 
states as relatively permanent conditions. 

I do not wish to close this chapter without indicating that 
ergodic theory is a considerably wider subject than we have 
indicated above. ‘There are certain modern developments of 
ergodic theory in which the measure to be kept invariant under 
a set of transformations is defined directly by the set itself, 
rather than assumed in advance. I refer especially to the work 
of Kryloff and Bogoliouboff, and to some of the work of Hure- 
wicz and the Japanese school. 

The next chapter is devoted to the statistical mechanics of 
time series. This is another field in which the conditions are 
very remote from those of the statistical mechanics of heat engines 
and which is thus very well suited to serve as a model of what 
happens in the living organism. 


CHAPTER III 


TIME SERIES, INFORMATION, 
AND COMMUNICATION 


There is a large class of phenomena in which what is observed 
is a numerical quantity, or a sequence of numerical quanlities, 
distributed in time. The temperature as recorded by a conti- 
nuous recording thermometer ; or the closing quotations of a 
stock in the stock market, taken day by day; or the complete set of 
meteorological data published from day to day by the Weather 
Bureau ; are all time series, continuous or discrete, simple or 
multiple. These time Series are relatively slowly changing, and 
are well suited to a treatment employing hand computation or 
ordinary numerical tools such as slide-rules and computing 
machines. Their study belongs to the more conventional parts 
ofstalistical theory. 

What is not generally realized is that the rapidly changing 
Sequences of voltages ina telephone line or a television circuit 
or a piece of radar apparatus belong just as truly to the field of 
statistics and time series, although the apparatus by means of 
which they are combined and modified must in general be very 
rapidin its action, and in fact must be able to put out results pari 
passu with the very rapid alterations of input. These pieces of 
apparatus — telephone receivers, wave filters, automatic sound — 
coding devices like the Vocoder of the Bell Telephone Labora- 
tories, frequency-modulating networks and their corresponding 
receivers — are all in essence quick-acting arithmetical devices, 
corresponding to the whole apparatus of computing machines 
and schedules, and the staf of computers, of the statistical 
laboratory. The ingenuity needed in their use has been built 
intothemin advance, just asit has into the automatic range-finders 
and gun-pointers of an anti-aircraft fire control system; and for 
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the same reasons. The chain of operation has to work too 
fast to admit of any human links. 

One and all, time series and the apparatus to deal with them 
whether in the computing laboratory or in the telephone circuit, 
have to deal with the recording, preservation, transmission, and 
use of information. What is this information, and how is it 
measured? One of the simplest, most unitary forms of informa- 
tion is the recording of a choice between two equally probable 
simple alternatives, one or the other of which is bound to 
happen — a choice, for example, between heads and tails in 
the tossing of acoin. We shall call a single choice of this sort 
a decision. If then we ask for the amount of information in the 
perfectly precise measurement of a quantity known to lie between 
A and B, which may with uniform a priori probability lie any- 
where in this range, we shall see that if we put A=0 and B=], 
and represent the quantity in the binary scale by the infinite 
binary number . dy dg «+ An ++, Where Gj, G2. each has the 
value 0 or 1, then the number of choices made and the 
consequent amount of information is infinite. Here 


a 1 
(BOL) gatas Cereaaen’ Qravesnnx = za +o Mgt vet ay Ant + 


However, no measurement which we actually make is per- 
formed with perfect precision. If the measurement has a uni- 
formly distributed error lying over a range of length by bo... Dn. 
where b, is the first digit not equal to 0, we shall see that all the 
and possibly to aj, are significant, 


decisions from a, to ai—1, gO 
The number of decisions 


while all the later decisions are not. 
made is certainly not far from : 


(Gi Gaicss On = 
(3.02) logs 5 5, 6, ie . 


and we shall take this quantity as the precise formula for the 


amount of information and its definition. of. 
conceive this in the following way: we know a priori 
nd 1, and a posteriori that it lies 
he amount of informa- 


edge is 


We may 
that a variable lies between 0 a 
on the interval (a,b) inside (0,1). Then t 
tion we have from our a posteriori knowl 

measure of (a, D) 
6.08) logs measure of (0,1) 
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However, let us now consider acase where our a priori knowledge 
is that the probability thata certain quantity should lie between 
wand x+ dz isf,(x) dx,and the a Posteriori probability is f,(x) dx. 
How much new information does our a posteriori probability 
give us ? 

This problem is essentially that of attaching a width to the 
regions under the curves y=/,(x) and y=/,(x). It will be noted 
that we are here assuming the variable to have a fundamental 
equipartition : that is, our results will not in general be the 
same if we replace a by x3, or any other function of a. Since 
[(x) is a probability density, we shall have : 


(3.04) hh * fi (@) dx = 1, 


so that the average logarithm of the breadth of the region under 


/;() may be considered as some sort of average of the height of 


the logarithm of the reciprocal of /\(x). Thus a reasonable mea- 
sure ' of the amount of information associated with the curve 


fi(x) is : 
(3.05) i (logs f(a) f,(w) da. 


—x 


The quantity we here define as amount of information is the 
negative of the quantity usually defined as entropy in similar 
situations. The definition here given is not the one given by 
R. A. Fisher for statistical problems, although it is a statistical 


definition, and can be used to replace Fisher’s definition in the 
technique of statistics, 


In particular. 
elsewhere, 


(3.06) he (logs filx)) fila) de = = log, 


» if f(x) is a constant over (a, b), and is zero 


1 1 
baa FO pea 
Using this to compare the information 


that a point is in the 
region (0,1), we obtain for the measure of 


the difference, 


Ade u 1 
(3.07) 10g. F—] — log: 1 = log, p=a" 


1. Here the author makes use of a personal communication of J. von Neumann. 
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The definition which we have given for the amount of infor- 
mation is applicable when the variable x is replaced by a va- 
riable rangingover two or more dimensions. In the two-dimen- 
sional case f(a, y) is a function such that : 


(3.08) if de { r dy f(x, y) = 93 


—7 


and the amount of information is : 


(3.081) ( dx ie dy f(x, y) logs fi(2, y)- 
J 


Note that if is ofthe form and 


(3.082) { Aton if vy) dy, 


—2 


then 


(3.083) [ te i dy (x) ¥(y) = 0; 


and 
(3.084) f “ae [dy Ale, 9) logs fe. 9) 
=f * dx 9x) loge ox) + J dy ey) loge ¥y) ; 


and the amount of information from independent sources is 
additive. 
An interesting problem is tl 


gained by fixing one or more variables in aproblem. Forexam- 
ble wu lies between x and x+dx 


nat of determining the information 


ple, let us suppose that a varia 


With the probability o aan dx, while a variable v lies bet- 
“ V¥ 2na 


= Lye Sees 
Ween the same two limits with a probability Vonks a de. 


How much information do we gain concerning U if we know 
that w+» = w? In this case, it is clear thatu=w—»v, where 
wis fixed. We assume the a priori distributions of w and v to 
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be independent. Then the a posteriori distribution of w is pro- 
portional to 


ae f=) —(w—c,)2 eh 
(3.09) @ %@e % é; “ 2abs 


where ¢, and ¢, are constants. They both disappear in the for- 
mula for the gain in information given by the fixing of w. 

The excess of information concerning x when we know w to be 
that which we have in advance is 


; 41 oe 244d 1 ab dl . 

—— (z—c,)? <2” ae, —(x—c)2—— j|dax 
aaa ee Fai ( 5 10g.2 ag aoe 

_— zt 2 1 a+b 

Vn J © #(—F logs 2na— -) dar = Flog SF 


—» 


Note that this expression (3.091) is positive, and that it is inde- 
pendent of w. It is one-half the logarithm of" the ratio of the 
sum of the mean squares of and v, to the mean square of 
v. If v has only a small range of variation, the amount of 
information concerning u which a knowledge of w+ v gives is 
large, and it becomes infinite as b goes to (). 

We can consider this result in the following light : let us treat 
uv as a message and v as a noise. Then the information carried 
by a precise message in the absence of a noise is infinile. In 
the presence of a Noise, however, this amount of information is 
finite, and it approaches () very rapidly as the noise increases 
in intensity, 

We have said that amoun 
logarithm of a quantity whi 
is essentially a neg. 
that, on the average 
entropy. 


t of information, being the negative 
ch we may consider as a probability, 
ative entropy. It is interesting to show 
» it has the Properties we associate with an 


Let 9(v) and ¥(x) be two Probability densities ; then se) 

is also a probability density. Then . 
* OX) + o(ax x } 

(3.10) ie aie log » a) dx 


—n 


_ a % 
<} f 2x) log < dx +f (x) log ae) ae 


—n 
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This follows from the fact that 
; P 
(3.11) log ae < log—~ + log 


In other words, the overlap of the regions under (x) and $(x) 
reduces the maximum information belonging to ox) + (2). 
On the other hand, if 9(x) is a probability densily vanishing 
outside (a, b) 


(3.12) a (2) log 9(x) dx 


—) 


over (a, b) and 1s zero elsewhere. 


. go 1 
isa minimum when 9(v)— — 


This follows from the fact that the logarithm curve is convex 
upward. 

It will be seen that the processes which lose information are, 
as we should expect, closely analogous to the processes which 
gain entropy. They consist in the fusion of regions of proba- 
bility which were originally distinct. For example, if we 
replace the distribution of a certain variable by the distribution 
of a function of that variable which takes the same value for 
different arguments, or if ina function of several variables we 
allow some of them to range unimpeded over their natural 
Tange of variability, we lose information. No operation on a 
message can gain information on the average. Here we have a 
Precise application of the second law of thermodynamics in 
Communication engineering. Conversely, the greater specifica- 
tion of an ambiguous situation, on the average, will, as we have 
seen, generally gain information, and never lose it. - 

An interesting case is when we have a probability distribution 
with n-fold density /(2,...,tn) over the variables (2j,..., nm), and 
where we have mn dependent variables Ys,---1 Ym- How: much 
information do we get by fixing these m variables ? First let 
them be fixed between the limits y;, Yr G13 +++» You Ym 1 dym. 
Let us take as a new set of variables Yu, +1 Ym» Mn+ 15 aman 
Then over the new set of variables, our distribution function 
will be proportional to f(@1, wap tn) over the region B given by 
y<y"* <y* + (dy)*, 05 Ym <Ym< ym + dy*» and 0 outside. Thus 
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the amount of information obtained by the specification of the 
y's will be : 


(3.13) vi dates, Pe Atty fly, ---5%n) lOgs fs, ---5n) 


Sf tes. f den [G35 50a) 


——_——_ 
R 


— dx... if Arn f(y, «+45 %n) logs flay, «..5 Xn) 


={ day... a i Yas sees Pm ) 
Tr—m-+-15+++35Xm 
jk dey... f (i Hee 


— fae... [ ae, (1, «.., Xn) logs Ons str) 


—o — 


f(X11..-50)logef(a,,...,7n) 


I ( Ye Y"m 
Tr—m+1 Lm 


came 


Closely related to this problem is the generalization of 
that which we discussed in (3.13); in the case just discussed 
how much information do we have concerning the variables 


Ty +++» n—m alone : Here the a priori probability density of these 
variables is : 


(3.14) i, AT n—m 44 iia Xn [iy .25Ln)3 


and the un-normalized probability density after fixing the y’’s is: 


(3.141) » 1( Yrs Ym 


Tr—m+15 0105 Ln 


) | fe, aeeen) 


where the 3 is taken over all Sets of points (@p—m +4, «+5 Un) COl- 
responding to a given set of y's. On this basis, we may easily 
write down the solution to our problem, though it will be a bit 
lengthy. If we take the set (1, -+.52n—m) to be a generalized 
message ; the set (tn—m441,..., tn) to be a generalized noise ; and 
the y”’s to be a generalized Corrupted message, we see that we 


haye given the solution of a generalization of the problem 
of (3.141). 
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We have thus at least a formal solution of a generalization of 
the message-noise problem which we have already stated. A 
set of observations depends in an arbitrary way on a set of 
messages and noises with a known combined distribution. We 
wish to ascertain how much information these observations give 
concerning the messages alone. This is a central problem of 
communication engineering. It enables us to evaluate different 
systems, such as amplitude modulation or frequency modulation 
or phase modulation, as far as their efficiency in transmitting 
information is concerned. This is a technical problem and not 
suitable toa detailed discussion here ; however, certain remarks 
are in order. In the first place, it can be shown that with the 
definition ofinformation given here, with a random «static» on the 
ether equidistributed in frequency as far as power is concerned, 
and with a message restricted to a definite frequency range 
and a detinite power output for this range, no means of trans- 
mission of information is more efficient than amplitude modu- 
lation, although other means may be as efficient. On the other 
hand, the information transmitted by this means is not neces- 
sarily in the form most suitable for reception by the ear or by 
any other given receptor. Here the specific characteristics of the 
ear and of other receptors must be considered by employing a 
theory very similar to the one just developed. In general, the 
efficient use of amplitude modulation or any other form of 
modulation must be supplemented by the use of decoding 
devices adequate to transforming the received information into 
a form suitable for reception by human receptors or use by 
mechanical receptors. Similarly, the original message mie ke 
Coded for the grestest compression in transmission. is 
problem has been attacked, at least in part, In the design of the 
“vocoder » system by the Bell Telephone Laboratories, and 
the relevant general theory has been presented in avery satis- 
factory form by Dr. C. Shannon of those laboratories. ; 

So much for the definition and technique of sess 
information. We shall now discuss the way 1n pvc infor- 
mation may be presented in a form homogeneous 1n time. Let 
itbe noted that most of the telephone and other communication 
devices are actually not attached to a particular origin in time. 
There is indeed one operation which seems to contradict this, but 
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which reallydoes not. This is the operation of modulation. This, 
in its simplest form, converts a message /(/) inlo one of the form 
/() sin (at+ b). If however we regard the factor sin (af + b) as 
an extra message which is put into the apparatus, it will be seen 
that the situation will come under our general statement. The 
extra message, which we call the carrier, adds nothing to the 
rate at which the system is carrying information. All the 
nformation it contains is conveyed in an arbitrarily short 
interval of time, and thereafter nothing new is said. 

A message homogeneous in time, or as the statisticians call 
it, a lime series which is in statistical equilibrium, is thus a 
single function or a set of functions of the time, which forms 
one of an ensemble of such sets with a well-defined proba- 
bility distribution, not altered by the change of f to ¢ +7 
throughout. That is, the transformation group consisting of the 
operators T” which change /(¢) into /(t + 2) leaves the probabi- 


lity of the ensemble invariant. The group satisfies the properties 
that 


(3.15) TTe f(t))=Te f(t) = (— wo << ,— 00 <p < 0). 


It follows from this that if P{ f(t) {is a « functional » of f() — 
that is, a number depending upon the whole history of /(f)—and 
1 the average of f(t) over the whole ensemble is finite, we are 
in a position to use the Birkhoff ergodic theorem quoted in the 
last chapter, and to come to the conclusion that, except for a set 
of values of /(0) of zero probability, the time-average of | /(t){,0F 
in symbols, 


. ii 7A { \ . 
(3.16) fim — J ®}no+94 ar. Lim 1 (pute de 
a \ Ay % 
—A 
exisl|s. 


There is even more here than this. We have stated in the 
last chapter another theorem of ergodic character, due to yon 
Neumann, which states that except for a set of elements of zero 
probability, any element belong, 


ing to a system which goes into 
itself under 


a group of measure-preserving transformations 
such as (3.15), belongs to a sub-set (which may be the whole set) 
which goes into itself under the same transformation, which has 
a measure defined over itself and also invariant under the 
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transformation, and which has the further property that any 
portion of this sub-set with measure preserved under the group 
of transformations either has the maximum measure of the sub- 
set, or measure 0. If we discard all elements except those of 
such a sub-set, and use its appropriate measure, we shall find 
that the time average (3.16) is in almost all cases the average of 
| f(t) | over all the space of functions /(/) ; the so-called phase 
average. Thus in the case of such an ensemble of functions /(t), 
except in a set of cases of zero probability we can deduce the 
average of any statistical parameter of the ensemble — indeed 
we can simultaneously deduce any countable set of such para- 
meters of the ensemble — from the record of any one of the 
component time series, by using a time average instead of aphase 
average. Moreover, we only need to know the past of almost 
any one timeseries of the class. In other words, given the entire 
a lime series known to belong to an 


history up to the present, of ‘ 
we can compute with 


ensemble in statistical equilibrium, 
probable error zero the entire set of statistical parameters of an 
ensemble in statistical equilibrium to which that time series 
belongs. Up to here, we have formulated this for single time 
Series ; it is equally true, however, for multiple time series, 1n 
which we have several quantities varying simultaneously, rather 
than a single varying quantity. 

We are now ina posilian to discuss various problems be- 
longing to time series. We shall confine our attention to those 


cases where the entire past of a time series can be given in 
terms of a countable set of quantities. For example, for quite a 
wide class of functions fil) (—* <! <)> We have fully deter- 
mined f when we know the set of quantities : 


(3.17) = fre in f(d) dt n= (0, 1,2, :0) 


—2 


Now let A be some function of the values of ¢t in the future: 


that is, for arguments greater than (. Then we can dletenine 
the simultaneous distribution of (40, Gv» 4 Ay om the past 
of almost any single time series, if the set of f s is taken i i. 
narrowest possible sense. In particular, if Agse«+s a are 2 

given, we may determine the distribution of A. Here we ap- 
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peal to the known theorem of Nikodym on conditional proba- 
bilities. The same theorem will assure us that this distribution, 
under very general circumstances, will tend to a limit as 
n— © and this limit will give us all the knowledge there is con- 
cerning the distribution of any future quantity. We may 
similarly determine the simultaneous distribution of values of 
any set of future quantities, or any set of quantities depending 
both on the past and the future, when the past is known. If 
then we have given any adequate interpretation to the « best 
value » of any of these statistical parameters or sets of statistical 
parameters — in the sense, perhaps, of a mean or a median or 
a mode — we can compute it from the known distribution, 
and obtain a prediction to meet any desired criterion of good- 
ness of prediction. We can compute the merit of the predic- 
tion, using any desired statistical basis of this merit — mean 
Square error or maximum error or mean absolute error, and 
soon. We can compute the amount of information concerning 
any statistical parameter or set of statistical parameters, which 
fixing of the past will give us. We can even compute the 
whole amount of information which aknowledge of the past will 
give us of the whole future beyond a certain point ; although 
when this point is the present we shall in general know the 
latter from the past, and our knowledge of the present will con- 
tain an infinite amount of information. 

Another interesting situation is that of a multiple time series, 
in which we know precisely onl ythe pasts of some of the com- 
ponents. The distribution of any quantity involving more than 
these pasts can be studied by means very similar to those al- 
ready suggested. In particular, we may wish to know the dis- 
tribution ofa value of another component, ora set of values of 
other components, at some point of time, past, present or future. 
The general problem of the wave-filler belongs to this class- 
We have a message, together with a noise, combined in some 
way intoa corrupted message, of which we know the past. We 
also know the statistical joint distribution of the message and 
the noise as time-series. We ask for the distribution of the 
values of the message at some given time, past, present, and 
future. We then ask for an operator on the past of ihe cor- 
rupted message which will best give this true message, in some 
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given statistical sense. We may ask for a slatistical estimate 
of some measure of the error of our knowledge of the message. 
Finally, we may ask for the amount of information which we 
possess concerning the message. 

There is one ensemble of time series which is particularly 
simple and central. This is the ensemble associated with the 
Brownian motion. The Brownian motion is the motion of a 
particle in a gas, impelled by the random impacts of the other 
particles in a state of thermal agitation. The theory has been 
developed by many writers, among them Einstein, Smoluchow- 
ski, Perrin, and the author?. Unless we go down in the time 
scale to intervals so small that the individual impacts of the 
particles on one another are discernible, the motion shows a 
curious kind of undifferentiability. The mean square motion 
in a given direction over a given time is proportional to the 
length of that time, and the motions over successive times are 
completely uncorrelated. This conforms closely to the physical 
observations. If we normalize the scale of the Brownian mo- 
tion to fit the time scale, and consider only one coordinate « of 
the motion, and if we let (¢) equal 0 for i=0, then the proba- 
bility that if0 <i;<&<-- < tp the particles lie belween w,and 
x, + da, at time f,...., between tm and a, + dan at time tn, is. 


ai (wa 2) Gna) 
(3.18) ( 2 (ts—4) Ain—bn—) 1 dey... dan. 
GE 


Vo (2n)" h(ta—1) -- 


On the basis of the probabili 
which is unambiguous, we can 
ding to the different possible 


ty system corresponding to this, 
make the set of paths correspon- 
Brownian motions depend on a 


Parameter «a lying between 0 and 1, in such a way that each 
path is a function x(t, «), where x depends on the time and the 
Parameter of distribution «, and where the probability that a 
Path lies in a certain set S is the same aS the measure of the aet 
of values of « corresponding to paths in S. On this basis, 
almost all paths will be continuous and non-differentiable. 

A very interesting question is that of determining the average 
with respect to « of x(L,a)...0(fn2)- This will be: 
ourier Transforms in the Complex 


1. Paley, R d Wiener N., « F 
ey, R. E. A. C. an lener Final Chapter. 


Plane ». American Math. Soc. Col. Pub. (1933), 
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1 
(3.19) de x(ti,«) x(to,0) ... 2(tn,«) 
e 


n 


1 

=(2) 2 [Alj—t)..(tn—tr—)] 2 
3 eee of = (ae —) 
i di. Es Ea Ban sein 3 2a, 2(te—a,) ~~ 2(atm—2n-1) 


under the assumption that0 <t,<...<%. Let us pul: 


(3.20) Eye. En = SAn Gy" Es — Fy) 62... (En — Ent), 


where ’i,1 + 24,2 + ... Aen =n. Then if we put & =0 the expres- 
sion of (3.19) will become : 


3.21) YAvQx)—F] tPA(t, PR —tn—a)in | 


s boo aT 
I ‘3 de ete Wr 
Y ey ’ 


vol = 


= ei 
o 1 ” Pie * @ eh 7 
x Bree d&(tj — fj_1) 
x i V2n Ss 


| O if any 9; is odd; 


Il 


shat 


S At T[ Qn; —1) (cj — 8) ...5. 3.(t;—G4) 2 
k 7 
if every x; is even, 


= X Ax [[ (number of ways of dividing 2,,; terms inlo 
- 


n ej 
pairs) x (jt) 4" 
= § Ax (numbers of ways of dividing n terms into pairs 
k 


whose elements both belong in the same | 
group of %,j terms into which 2 is separated) 


alps 
(tj — ta) 3 


= dAj Sul J ala (te, «) — (ty—1, @) ) 


* (@(lgs @) — aly, 2) ). 
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Here the first © sums over j; the second, over all the ways of 
dividing n terms in blocks respectively, of 44,1 ,..-) 4xn numbers, 
into pairs; and, the = is taken over those pairs of values k and 
q; where };,1 of the elements to be selected from fx and é, are 
th, dng are fz, and so on. It immediately results that: 


‘1 
(3.22) da (ty, @) @(ty, #)..-. @(n, «) 
f 


= alll te a(t, «) (tk, %) 5 
y 


where the © is taken over all partitions of t,..., tn into distinct 
pairs, and the x over all the pairs in each partition. In other 
words, when we know the averages of the products of 2x(éj,«) 
by pairs, we know the averages of all polynomials in these 
quantities, and thus their entire statistical distribution. 

Up to the present, we have considered Brownian motions 
x(t,«) where ¢ is positive. If we put 
(et,,8) = aha) E20); 
L(t, «, p=) =2(—t 6) (fF <0). 


where « and shave independent uniform distributions over (0,1), 
_We shall obtain a distribution of &(/, « 8) where f runs over the 
whole real infinite line. There is a well-known mathematical 


device to map a square on a line segment in such a way that 
area goes into length. All we need to do is to write our coor- 
dinates in the square in the decimal form : 


(3.23) 


(3.24) = witty these: Crise BP Se By Ba. Bae 5 

and to put : 
yay By %2 Bg--ess On Bnewes > 

and we obtain a mapping of this sort which is one-one for 
almost all points both in the line-segment and the square. 
Using this substitution, we define : 

(3.25) E(tyy) = El, % ®)- 

We now wish to define : 


(3.26) fe ‘ K(t) d&(l, y)- 


—20 
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The obvious thing would be to define this asa Stieltjes integral; 
but & is a very irregular function of t, and does not make such 
a definition possible. If, however, K runs sufficiently rapidly to 
zero at + coand is a sufficiently smooth function, it is reason- 
able to put : 


(3.27) if “Rae = — J “K(t) Ey) dt. 


Under these circumstances, we have formally : 


(3.28) af Sie fi “Ku de(t.) wi “Ky(t) de(t.) 


0 


wk co] et 
= [a Ki) tx) dt f Ki(e ety) dt 
Vas i 
co, C) 1 
= Kis) ds Ki(t) dt &s,7) E(t) dy 
ay Osi 


Now, ifs and ¢ are of Opposite signs, 


1 
(3.29) Sf HS.r) (by) dy = 0 
0 


while if they are of the same sign, and |s|<| t| 


(3.30) yf E(ssx) E(t) tre fxishe x(| t|,«) da 
u? v—u)?* 
~ nVist( Ar aves @ tiff a Bal Ripa) 
ee ee 2e Sal 
a |s| ie * au 


¥ 
=i f we? du=|s|. 
ud 


1, Stieltjes, Annales de la Fac. des Sc. de Toulouse, 1894; Lebesgue 


sur l'Intégration. Paris, 1928. H., Lecons 
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a ‘dr yp K,() dete) y. ne 
(3.31) =—— f * Ks) ds ve “KA at — is Kis) ds up "1K at 
+ Ki(s)ds is 1K) dt-+ ye KG) ds ue EK '(Oat 
2s i Ki(s)ds ( sKs(s)— = Kat) J 86928 (0 4) ‘knat) 
+ fia —sKo- f Kndt + ff Ki(onds(—sk)— f° "Kat ) 
¥ ¥ fsa) a aif me ds. | 


In particular, 
(3.32) Gh ue Why K(t + 5) d&(t, ») aA K(t-+) det 7) 
0 —~o _—o 


— uh K(s) K(s++t2 — 1) ds. 


Thus: 


Moreover, 


(3.33) fi Sale II i 4 K(t + +) d &(t,y) 
0 =1 —% 
= » Ul va K(s) K(s +77 — an) ds, 


where the sum is over all partitions of 7... +» into pairs, and 
the product is over the pairs in each partition. 
The expression : d 


(3.34) a OKChel bets = TO) 


represents a very important ensemble of time series in the 
variable ¢, depending on a parameter of distribution vy. We 
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have just shown what amounts to the statement that all ihe 
moments and hence all the statistical parameters of this distri- 
bution depend on the function: 


(3.35) B(x) = a K(s) K(s-++) ds 


_ J K(s-+ 2) K(s +14 =) ds 


ce) 


which is the autocorrelation function of the statisticians with 
log y. Thus the statistics of distribution of /(t, ~) are the same 
as the statistics of f(t-+t, y); and it can be shown in fact that if 


(3.36) Ki+b, x) = fit, 0) 


then the transformation of y into T preserves measure. In other 
words, our time-series f{( y) is in statistical equilibrium. 
Moreover, if we consider the average of 


(3.37) uf K(t—z) dé (t, yf KU +o—9 dé (t)) 
it will consist of precisely the terms in 


1 ee m 1 = ie 

(3.38) oh ah fs K(—=)ae(t)) £ anf Sf K(l-+e—s)de(t) ) 

0 —~7 0 —~« 

together with a finite number 
powers of 


(3.39) ve K(o+4- ) K(x) de ; 


and if this approaches 0 when c->24, 
(3.37) under these circumstances. 

f(t+ 4 y) are asymptotically indep 
aso—>co. By a more gener 


of terms involving as factors 


(3.38) will be the limit of 
In other Words, f (¢, y) and 
endent in their distributions 
ally phrased but entirely 


: similar 
argument, it may be shown that the simultaneous distribution of 
fly Dre Mls x) and of fle + 1, +),... fic + §m, +) tends to the 


joint distribution of the first and the sec 
independent other words, any bounded measur 
quantity depending on the entire distribution 


ondiset as 's-s.05. In 


able functional or 
of the values of the 
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function of t, f(é,v), which we may write in the form #} f(t, y)|, 
must have the property that 


. 1 2 
(3.40) Lim Sf Fine Orin, aidr=[_fFIKtoar] 
o>n 7 0 


If now #} f(t, y){ is invariant under a translation of ¢, and only 
takes on the values 0 or 1, we shall have: 


(3.41) [3 ‘(ty i dr= [/ sin x) iar] 


0 0 
so that the transformation group of f (i, v) into {(t + 7) is metri- 


It follows that if*}/(é,y) {is any integrable 


cally transitive. 
s a function of ¢, then by the ergodic theorem, 


functional of fa 


(3.42) x 'g \ f(t, v) 4 dy 


0 


1 7 
lim > | * | (va) | at 


Dd 


ll 


I 


0 
Lim = F} f(t), v1) | at 


tT 2 
— 


for all values of y, except for a set of zero measure. That is, we 
can almost always read off any statistical parameter of such a 
time series, and indeed any denumerable set of statistical para- 
meters, from the past history of a single example. Actually, for 


such a time series, when we know 
(3.43) lim = Sf few) fis) at, 
Tow 
= 


we know ®(é) in almost every case, and we have a complete 


statistical knowledge of the time series. 
There are certain quantities dependent 
sort which have quite interesting properties. In particular, it 


is interesting to know the average of: 


(3.44) exp. (i J * K(e) dety) ) 


on a time series of this 
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Formally, this may be written : 
4 C le co) n 
45 y d&(t,y) ) . 
(3.45) yi dy x =(f K(d) d& ») 
0 B= — 


<1 / re : 
5 te = es K® )* at) (2m —1) (2m—3) ... 5.3.1 
_ See eee 
= Vigan Le (KW) at) 


exp (— sf Kw ; at). 


It isa very interesting problem to try to build up a time series 
as general as possible from the simple Brownian motion series. 
In such constructions, the example of the Fourier developments 
suggests that such expansions as (3.44) are convenient building 


blocks for this purpose. In particular, let us investigate time 
series of the special form : 


b cy 
(3.46) uf da exp (i yy Kil + 5. des), 


Let us suppose that we know E(x 
Then as in (3.45), if t; > te 


‘1 b i} 
(3.47) fh dy ets(8(t7)—8(ts,y) fe dxexp (i of KU + =)ds(t) ) 
0 a —2 
= [drex ut “(KC 2 a a 
Ps (t-+7,2)) d=) —s y K¢tayat), 
a =n, ty 


s(ts—t) 
Ifwe now multiply bye 2 
we obtain : 


(3.48) ff “dnexp(— = fe “WEE, »))? dt — ie K(t,, »)) 


Let us take K(t, 
a, obtaining 


(3.49) a= Q(t, u). 


i} 


»Y) as well as expression (3.46). 


let s(t, — tf) = ie and let t2= th, 


a), a new independent Variable u and solve for 
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Then (3.48) becomes : 
K(t,6) hy» 1 2 
(3.50) zs eins ate! exp (-5 de (K(i—+, Q(t, jrat), 


K(t,4) 


From this, by a Fourier transformation, we can determine : 
1 20 
(3.51) Sete *) exp (-3 he (K(t— + Qh, wyedt) 
13 4 


as a function of » when v lies between K(t,, a) and K(&, b). If 


we integrate this function with respect to » we determine 


(3.52) fiw exp (- bf wi —- 7, a))? at), 


as a function of K(4,) and t,. That is, there is a known func- 


tion F(u, v), such that 


(3.53) nf “D exp (— > ve ie a)? at) = F(K(é, 2), 4): 


0 
Since the left-hand side 
we may write it G(a), and put: 
(3.54) F(K(L, 9) 4) = GQ). 
e can invert it with respect 


of this equation does not depend on hy, 


Here, F is a known function, and w 
to the first argument, and put : 


(3.55) K(4,2) = H(GQ), 4), 


where it is also a known function. Then: 


C) 1 S 
(3.56) Go= f dd exp a of [H(G(a), pjrat.) 


Then the function 


(3.57) exp = sf (H(u, 2)? at) = R(u) 


will be a known function, and 


(3.58) = R(G). 
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That is, 
dG 
5 sm = da, 
(3,59) RG 
or 


(3.60) ve f ae + const. = S(G) + const. 


This constant will be given by: 


(3.61) G(a) = 0, 
or 
(3.62)- a = S(o)+ const. 
It is easy to see that if @ is finite, it does not matler what value 


we give it; for ouro 
to all values of x. 

determined %, as 
of 2. 


perator is not changed if we adda constant 
We can hence make it 0. We have thus 
a function of G, and thus G as a function 
Thus, by (3.55), we have determined K(i,2). To finish 
the determination of (3.52), we need only know b. This can be 
determined, however, by a comparison of 


(3.63) ie * divexp (- > f “EK(ta)p at) 


with 


1 b 2 
(3.64) dy dx exp (i K(d,a) d&(t,y) ). 
Jf efiecso(: f 


Thus, under certain circumsta 


nces which remain to be defini- 
tely formulated, if a time seri 


es may be written in the form 
(3.52), and we know E(t.,) as well, we can determine the function 


K(4,A) in (3.52) and the numbers a and b, except for an 
undetermined constant added lo a, i, and b. There is no extra 


difficulty if b = + co and itis not hard to extend the reasoning 
to the case where a = — x of course, 


A good deal of work 
remains to be done to discuss the Problem of the inversion of 
the functions inverted when the results are not Single valued, and 
the general conditions of validity of the €xpansions concerned. 
Still, we have at least taken a first step 
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formal application of the theories of prediction and of the 
measurement of information, as we have sketched them earlier 
in this chapter. 

There is still one obvious limitation which we should remove 
from this approach to the theory of time series: the necessity 
which we are under of knowing &(t,y) as well as the times series 
which we are expanding in form (3.46). The question is: under 
what circumstances can we represent a time series of known 
statistical parameters as determined by a Brownian motion; or 
at least as the limit in some sense or other of time series deter- 
mined by Brownian motions? We shall confine ovrselves to 
time series with the property of metrical transitivity, and with 
the even stronger property, that if we take intervals of fixed 
length, but remote in time, the distributions of any functionals 
of the segments of the time series in these intervals approach 
independence as the intervals recede from each other *. The 
theory to be developed here has already been sketched by the 
author. 

If K(t) is a sufficiently continuous function, it is possible to 


show that the zeros of 


(3.65) Fe K(t + +) dé (*5Y) 


almost always have a definite density, by a theorem of M. Kac, 
and that this density can be made as great as we wish by a proper 
choice of K. Let Ky be so selected that this density is D. Then 


the sequence of zeros of f K, (t—*) dé (s, y) from —% to % 


will be called Zn (D, y), -e<n< &%. Of course, in the numera- 
tion of these zeros, n is only determined except for an additive 
constant integer. 

Now, let T(é, ») be any time series in the continuous variable 
1, while » is a parameter of distribution of the time series, vary- 
ing uniformly over (0,1). Then let 


(3.66) Tp (4, # y) = T (t —Zn(D, ¥)), 


1. This is the mixing property of Koopman, which is the necessary and suffi- 
cient ergodic assumption to justify statistical mechanics. 
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where the Z, taken is the one just precedingt. It willbe seen that 
for any finite set of values t,, 1, .... ty of a, the simultaneous dis- 
tribution of T, (tx, vu, y) (K = 1, 2,...v) will approach the simulta- 
neous distribution of T(t, y) for the same ixs as D—> o, for 
almost every value of p. However, T, (t, », y) is completely 
determined by f,n, D and & (=, x). It is therefore not inappropriate 
to try to express T, (i, p, y), for a given D and a given », either 
directly in the form (3.52), or in some way or another as a 
time series which has a distribution which is a limit (in the 
loose sense just given) of distributions of this form. . 

It must be admitted that this is a program to be carried 


through in the future, rather than one which we can consider as 
already accomplished. Nevertheless 


in the Opinion of the author, offers th 
consistent treatment of the many 


, it is the program which, 
e best hope for a rational, 
problems associated with non- 
linear prediction, non-linear filtering, the evaluation of the 
transmission of information in non-linear situations, and the 
theory of the dense gas and turbulence. Among these problems 
are perhaps the most pressing facing communication enginee- 
ring. 

Let us now come to the prediction problem for time-series of 
the form (3.34). We see that the only independent statistica! 
parameter of the time series is @(t), as given by (3.35) ; which 
means that the only significant quantity connected with K(t) is 


(3.67) if K(s) K(s + t) ds. 
Here of course K is real. 
Let us put 
(3.68) K(s) = Vee K(a) eiws de, 


employing a Fourier transformation, 
k() and vice versa. Then 


G69) = KW) Ks +2) a= 5 Ove ee 


To know K(s) is to know 
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Thus a knowledge of ©() is tantamount to a knowledge of 
k(e) k(—e). Since however K(s) is real, 


(3.70) K(s) = vi K(@) e-#s da, 
whence k(o) = k( — o). Thus | k@)|? is a known function, 


which means that the real part oflog| k(@)| is a known function. 
If we write 
(3.71) F() = R log (K(@) ), 

then the determination of K(s) is equivalent to the determainalion 
of the imaginary part of log (k(e) ). This problem is not 
determinate unless we put some further restriction on k(#). The 
type of restriction which: we shall put is that log k(o) shall be 
analytic and of a sufficiently small rate of growth for in the 
upper half plane. In order to make this restriction, k() and 
(k(w))-1 will be assumed to be of algebraic growth on the real 
axis. Then (F(«))* will be even and at most logarithmically 
infinite, and the Cauchy principal value of 


(3.72) G(e) = f° FO au 


u— oe 


will exist. The transformation indicated by (3.72), known as 
the Hilbert transformation, changes Cos 4 into sin A and sin do 
into — cos rx. Thus F() + iG() is a function of the form 


(3.73) we * iw d(MQ) ) 
0 
and satisfies the required conditions for log | k(o)| in the lower 
half plane. If we now put 
(3.74) k(o) = exp. (Fle) + § Gla), 
it can be shown thal k(@) is a function which, under very 


general conditions, is such that K(s),as defined in (3.68), vanish- 
es for all negative arguments. Thus 


(3.75) Kt) = ‘K(t + +) d&(z, 7) 
4 
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On the other hand, it can be shown that 


we may write 1/k(«) in 
the form 


(3.76) lim ee d'N,(a), 
oe 
where the N's are properly 
done in sucha way that 


(3.77) &(, 7) = lim ‘ds “On(t + s) [(c, y) do. 
n—-n Jf J 


Here the Q/s must have the formal property that 


(3.78) f(t) = lim “ke + 2) d= y "Biot ed ds. 


determined ; and that this can be 


In general, we shall have : 


(3.79) 4) = lim ‘a K(t ++) de a “Qitrt0) 40) ds, 


or if we write (as in (3.68) ) : 


(3.80) K(s) = a “Klee ay 
Qnr(s) -{" Arla) eimsd ey 
Ts) = ve “OR elie: re 
then: "7 
C81 FO) = him ny 66) g6(—oy Ko) 
Thus: 
(3.82) lim qu(—e) = 1 


We shall find this result useful in 
tion into a form concerning frequency rather than time 

Thus the past and present of E(t.) (or properly a the 
« differential » d&(t, y)) determine the past and tsar cae I(t). 


Setting the ©perator of predic- 


and vice versa. 
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f “K(E+ A+) délzy) 


t—A 


f KG + A+) dts) 


=X 


+ f° K(t +A ++) d&(x,7). 


Now, if A>0, 


(3.83) fl + Aw) 


Il 
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Here the first term of the last expression depends on a range 
of d&(z,y) of which a knowledge of /(«,7) for «< # tells us nothing, 
and is entirely independent of the second term. Its mean 


square value is : 


(3.84) f- warararara fe) a 


—t-—A 


and this tells us all there is to know about it statistically. 


It 


may be shown to have a Gaussian distribution with this mean 
square value. It is the error of the best possible prediction 


of f(t-+ A,y). 


The best possible prediction itself is the last term of (3.83), 


(3.85) K (¢Q +A ++) dé (+, y) 
t : 
= Hin BS Kd +AtDaG | Qu (= +9) (ev) de. 


If we now put : 


(3.86) kk (@) = - K(t-+ A) e—iwt dl: 
7 


and if we apply the operator of (3.85) to et", obtaining: 


(3.87) Jim ve RG A+ )de f & (« +e) eiws do 


—t 
= A(a) eiws ; 


we shall find out (somewhat as in (3.81) thal : 
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(3.88) A (0) = BM (2299/2 gn (—a) ky (@) 
= k, (e)/k(o) 


1 20 26 
= —iw (t—a) dt k tut . 
Taka) f e d Ay k(@) eit du 


This is then the frequency form of the best prediction operator. 

The problem of filtering in the case of time-series such as (3.34) 
is very closely allied to the prediction problem. Let our message 
plus noise be of the form : 


(3.89) m(?t) + n(t) = ve K(x) d &(t — +, y); 
0 
and let the message be of the form: 


(3.90) m(t) = ah Qs) d (t— 2, ) + fi “R(s)d &(t =, 8), 


where y and o are distributed independently over (0,1). Then 
the predictable part of the m(t+-a) is clearly : 


(3.901) [G+ ads, 
m7) 
and the mean square error of prediction is : 
3.902 f“Ta@Rae + Jf Bora. 


Furthermore, let us suppose that we know the followings 


g quantities 
1 “l 
(3.903) byt) = a dy if d8 n(t + =) n(s) 
i) 0 


é: We CKO] AL + >) — Qt + =)) K@)— QQ) ae 
0 


+ f° oll +9) &@—0@) & 


It] 


tf Qt +9 QM d+ RUE 4.) R(z) dz 
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= f K+ 9K@a— f WIE + IKE) ae 
0 


a—ifiel 


+ feel evamecr fai arity 


(3.904) “>,,(2) = fe dy of ds m(|t| + =) m(z) dz 


-f ‘eieievar arr /Ruaironows 


(3.905) w= fdr fe dsm(t+7+) n(s) d= 
0 


i) 


1 1 
dy ds m(t + +) (m(=) +(t) ) — Pals) 
fef 
A ay I K(o-+t) dé(=—o,7) Ja Qle)aeesern) 
0 


— 944(7) 


I 


ll 


= fp “KU +2) OG de— Puls) 


The Fourier transforms of these three quantilies are respec- 


tively : 
2a(@) =| K(@) )2+|a(a)|?—a(e) ke) — k(o) qe) + | r(@) |? 3 


(3.906) a =| q(o) 2+ |r(@) 5 
Saale) = ko) Go) — 19(@) |*— Io) Ps 


1 a —iws ds 5 
[oat K(s) eis ds ; 


alo) = 5 f_ 9 em ass 


Ms = x. R(s) ets ds. 


That is, 
(3.908) Pir() + Pisl@) + (a) + Pxlo) 


where 


(3.907) 


= | ke) |?5 
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and — 
(3.909) q(e) k(e) = ®1() + Py,(o), 


where for symmetry we write: Py(o) = P,(e). We can now 
determine k(#) from (3.908) as we have detined k(o) before on 
the basis of (3.74). Here we put ®(¢) for® ,,()+Peo( £)4+-2 R(P,(t) )- 
This will give us: 


(3.910) q (o) = 2u (2) + ®y (©) : 
K(e) 
Hence d 
* Py (0) +P (0) 
(3.911) Qh = f a eit da; 
ae K(@) 


and thus the best determination of m(i), with the least mean 
square error is : 


oa 2 ~, 
(3.912) J ew ee itd 
IC (@ 
: a I 


Combining this with (3.89), and using an argumentsimilar to the 
one by which we obtained (3.88), we see that the operator on 
m(t) + n(l) by which we obtain the « best » representation of m(é), 
if we write it on the frequency scale, is : 


‘ . 1 ae mn (u) + @ (w) 
(8.913) e—iw (t—a) dt see ek) 
2x k(w) ta f k@ 


a 


eit du. 


—n 


This operator constitutes a chara cteri 
electricians know as a wave filter. 
the filter. It may be either positive 
tive, — a is known as the lead. The apparatus corresponding 
to (3.913) may always be constructed with as much accuracy as 
we like. The details of its construction are more for the spe- 
cialist in electrical engineering than for the reader of this book. 
They may be found elsewhere ?. 

The mean square filtering error (3.902) ma 
as the sum of the mean square filtering error 


Stic operator of what 
The quantity a@ is the lag of 
or negative; when it is nega- 


y be represented 
for in finite lag : 


1. We refer especially to recent papers by Dr. Y. W. Lee, 
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(3.914) ra [RG]? d:=%,,(0) — VA [Q(=)]? d= 
1 fe 1 ,,(0) + Pq,(o 
md foodie bf |e 


a Ko) 
cee sf | Pu(o) + Pale) 7 ) 
Qn 


2 


de 


\ a d 
) Purl ) Py (0) + Pp o(@) + ®5,(0) a Poq( 0) 9 (sy 


9y,(0) D,(o) 
<. a / Poy(@) ®,,(o) 
an P(e) + ®12(@) + ®,,(@) + Po(@) 


do . 


and a part dependent on the lag : 

e Ss © Py 1(@) + Polo). fe 
3.915 ic) 12 = dl fa ate at eit dol . 
(3.915) y [ Q(x) ]? di x a eo 


It will be seen that the mean square error of filtering is a mono- 
tonely decreasing function of lag. 

Another question which is interestingin the case of messages 
and noises derived from the Brownian motion is the matter of 
rate of transmission of information. Let us consider for simpli- 


city the case where the message and the noise are incoherent : 


that is, when 
(3.916) (0) = Py(o) = 0. 
In this case, let us consider : 


my = f Model); 


—7 


(3.917) 2 
att) = vi Ni) d I — +. 8) 


where y and 8 are distributed independently. Let us suppose 
we know m(t)-+n(é over (— A,A) ; how much information do 
we have concerning m(t) ? Note that we should heuristically 
expect that it would not be very different from the amount of 


information concerning 


(3.918) J “M (2) dé(t—7)> 
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which we have when we know all values of 


(3.919) JOA [Node —=9), 


where y and 8 have independent distributions. 
be shown that the nt 


Guassian distribution 
coeflicients, 


It can, however 
h Fourrier coefficient of (3.918) has a 
independent of all the other Fourier 
and that its mean Square value is proportional to 


(3.930) | Mo DAG Peele 


Thus by (3.09), the total amount of information available con- 
cerning M is 


n=—x» 


7 eo (ee n i“ FP 
<0 M(zje 4 g ws ads 
Gor Ft Ae (z) + +| f Noe 
and the time-der 


28 et 
i NG@)e* ds 
at 


nd sity of communication of energy 
divided by 2A. If now A —- 2, (8.921) See 


? 1 ) fim 7) eiur 
(8.922) 5 ve du logy b= (5) eiuz qd, 


is this quantity 


2 ao 
+ | Sf N(c) ei dz 


2 


also on the noise level, 

tion to the audiograms u 
and loss of hearing in a given indiviq 
frequency, the ordinate of lower bo 
the intensity of the threshold of 
may call the logarithm of the in 


Mount of hearing 
€re the abscissa is 
atau is the logarithm of 
tensit aensity ap eagle 
1LY of the ternal noise of 


ual, 
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the receiving system —and the upper boundary, the logarithm of 
the intensity of the greatest message the system is suited to handle. 
The areabetween them, which is a quantity of the dimension of 
(3.922), is then taken as a measure of the rate of transmission 
ot information with which the ear is competent to cope. 

‘The theory of messages depending linearly on the Brownian 
motion has many important variants. The key formulae are 
(3.88), (3.914) and (3.922) ; together, of course, with the definilions 
necessary to interpret these. There are a number of variants 
of this theory. First: the theory gives us the best possible 
design of predictors and of wave-filters in the case in which the 
messages and the noises represent the response of linear reso- 
nators to Brownian motions; but in much more general cases, 
they represent a possible design for predictors and filters. This: 
will not be an absolute best possible design, but it will minimize 
the mean square error of prediction and filtering, in so far as 
this can be done with apparatus performing linear operations. 
However, there will generally be some non-linear apparatus which 
givesa performance still better than that of any linear apparatus. 

Next, the time series here have been simple time series, in which 
a single numerical variable depends on the time. There are also 
multiple time series, in which a number of such variables depend 
simultaneously on the time ; andit is these which are of greatest 
importance in economics, meteorology, and the like. The com- 
plete weather map of the United States, taken from day to day, 
constitutes such a time series. In this case, we have to develop 
a number of functions simultaneously in terms of the frequency, 
and the quadratic quantities such as (3.35) and the | k(«) |? of the 
arguments following (3.70) are replaced by arrays of pairs of 
quantities — that is, matrices. The problem of determining k() 
in terms of | k() |?, in such a way as to satisfy certain auxiliary 
conditions in the complex plane, becomes much more difficult, 
especially as the multiplication of matrices is not a permutable 
operation. However, the problems involved in this multi-dimen- 
sional theory have been solved, at least in part, by Krein and 
the author. 

‘The multi-dimensional theory represents a complication of the 
onealready given. There isanother closely related theory which 
is a simplification of it. This is the theory of prediction, filter- 
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ing, and amount of information in discrete lime series. Such 
a series is a sequence of funcions f;, («) of a parameter «, where 
nruns over all integer values from —ze tose. The quantity @ is 
as before the parameter of distribution, and may be taken to run 
uniformly over (0,1). The time-series is said to be in statistical 
equilibrium when the change of n ton+v(v on integer). 

The theory of discrete time series is simpler in many respects 
than the theory of the continuous series. It is much easier, for 
instance, to make them depend on a sequence of independent 
choices. Each term (in the mixing case) will be representable 
as a combination of the previous terms with a quantity indepen- 
dent of all previous terms, distributed aniformly over (0,1), and 
the sequence of these independent factors may be taken to 

‘replace the Brownian motion which is so important in the con- 
tinuous case. 

If fn () is a time series in Statistical equilibrium, and it is 
metrically transitive, its autocorrelation coefficient will be : 


1 
(3.923) m= fo fnle) file) da 
0 


and we shall have: 


a ie 1 N 
(3.924) dy = Np Sette) fax) 


lim 1 
=N—> oo N+1 x f—1—m(«) fia) 


for almost all «. Let us put : 


1 = 
(3.925) P, = On Po) einw de ; 
or ¥ 
(3.926) P(e) = » @, einw, 
Let : 


(3.927) a log bo) = & Pa cos ne, 
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and let : 

(3.928), Go) = 2 + % Pa om. 
Let : 

(3.929) estu) = K(a). 


Then under very general conditions, k(#) will be the boundary 


value on the unit circle of a function without zeros or singular- 


ilies inside the unit circle, if is the angle. We shall have: 


(3.930) | k(w) |? = P(e). 
If now we pul for the best linear prediction of fn(«) with a lead 
of v, 

(3.931) x fn—(a) Wo, 


we shall find that : 


ies, Sue 1 
(8.932) Wy et = 9 Ga) 


48 


way 


Seite») / k(u) e~inu du. 
f J 


This is the analogue of (3.88). Let us note that if we put 


1 i j 
(3.933) ky = 3 f k(u) e~ive du, 


then 
- SS ky eit 
3.934) SWaewo =e se 
0 V ky elt 
v 


v1 
y hy eve 
= e—ivw| 1 = 
Sku ele 
0 


It will clearly be the result of the way we have formed k() 


that in a very general set of cases, 


1 x i 
=y ee, 
k(o) at fe 


we can put: 


(3.935) 
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Then (3.934) becomes : 


CYBERNETICS 


w—1 


(3.936) SY Wy elu = e—ivo (1 — ¥ hy eve Yaa om). 
0 0 0 


In particular, if » = if 
(3.937) 3, W, eu» = e-iw G- kX Gh am) 
0 i) 


or 


(3.938) Wa =— gai ky 


Thus for & prediction one step ahead, the best value for /;,(«) is: 


(3.939) ke d qn-t fra @ ; 

0 
and by a process of step-by-step prediction, 
entire problem of linear 
As inthe continuous case, tl 
by any method if 


(3.940 fan (a) = ie “Kk (n—+) dé (z, «), 


The transfer of the filtering problem from the continuous to 
the discrete case follows much the same lines of 
Formula (3.913) for the frequency char 
filter takes the form : 


(3.941) Tie Dea ii : 
=e 
where all the terms receive the 
tinuous case, except that all integ 
instead of from —20 to o and all sums on y are discrete sums, 
instead of integrals on ¢. The filters for discrete time eeries 
are usually not so much physically Constructible devices to be 
used with an electric circuit, as mathematical Procedures to 
enable statisticians to obtain the best results with statistically 
impure data. ‘ 
Finally, the rate of transfer of information by a discrete time- 
series of the form 


we can solve the 
prediction for discrete time series. 
his will be the best prediction possible 


argument. 
acteristic of the best 


®y,(u) + Po, (u) ei du, 


Same definitions as in the con- 


rals on wor y are from — xtox 
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(3.942) / M(n—+)dé t, x) 


—o 


in the presence of a noise 


(3.948) Fi N(n—*) d&(6,8), 


when y and 8 are indepe 


ndent, will be the precise analogue of 
(3.922); namely : 


ES 2 0 , 
’ M(x) ei de | + wa N(<) eis dz 
(3.944) x fa loge ~— woe 2 

Qn J—x | if N() et a.| 
where over (—*, =), 
(3.945) | i M(-) e" ae | 


er distribution of the message in frequency, 


2 


represents the pow 
and 


=—20 


(3.946) | 0 N(dz) ei: dz . 


that of the noise. } ; 
The statistical theories we have here developed involve a full 


knowledge of the pasts of the time series we observe. dn every 
case, we have to be content with less, as our observation does 
noi run indefinitely into the past. The development ofourtheory 
beyond this point, as a practical statistical theory, involves an 
extension of exisling methods of sampling. The author and 


others! have made a beginning in this direction. It involves 
all the complexities of the use, either of Bayes’ Law on the one 
hand, or of those terminological tricks in the theory of likelihood a 
on the other, which seem to avoid the necessity for the use of 
Bayes’ law, but which in reality transfer the responsibility e 
its use to the working statistician, Or the person who ultimately 


: ; istical theorist is quite 
employs his results. Meanwhile the statistical theo q 
iley, New York. 


1. Wiener, N. and Doob, forthcoming book, W 
2. See writings of. R. A. Fisher and J. von Neumann. 
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honestly able to say that he has said nothing which is not per- 
fectly rigorous and unimpeachable. 

Finally, this chapter should end with a discussion of modern 
quantum mechanics. These represent the highest point of the 
invasion of modern physics by the theory of time series. In 
the Newtonian physics, the sequence of physical phenomena is 
completely determined by its past, and in particular, by the de- 
termination of all positions and momenta at any one moment. 
Inthe complete Gibbsian theory it is still true that with a perfect 
determination of the multiple time series of the whole universe, 


the knowledge of all positions and momenta 
would determine the entire future. 


ignored, non-observed coordinates 
series with which we actually 


at any one moment 
It is only because these are 
and momenta that the time 
work take on the sort of mixing 
properly with which we have become familiar in this chapter, 
in the case of time series derived from the Brownian motion. 
The great contribution of Heisenberg to physics was the re- 
placement of this still quasi-Newtonian world of Gibbs by one 
in which the time series can in no way be reduced to an 
assembly of determinate threads of development in lime. In 
quantum mechanics, the whole past of an individual system 
does not determine the future of that system in any absolute 
way, but merely the distribution of possible futures of the 
system. The quantilies which the classical physics demands 
for a knowledge of the entire course of a system, are nol simul- 
taneously observable, except in a loose and approximate way, 
which nevertheless is sufficiently Precise for the needs of the 
classical physics over the range of’ precision Deve tiohasubeci 
shown experimentally to be applicable. The conditions of the 
observation of a momentum and ils Corresponding Pposilion are 
Incompatible. To observe the position ofa System as precisely 
as possible, we nish ODSE ive ut with light or electron-waves or 
similar means of high resolving Power, or short wave length. 
However, the light has a particle action depending on its fre= 
quency only,and to illuminate a body With high-fre 
means to subject it toa change in its Momentum 
with the frequency. On the other hand, 
light that gives the minimum change jy tl 
as not aq 


quency light 
which increases 
it is low-frequency 
1€ momenta of the 


particles it illuminates, and this h Sufficient resolving 
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power to give a sharp indication of positions. Intermediate 
frequencies of light give a blurred account both of positions and 
of momenta. [In general, there is no set of observations con- 
ceivable which can give us enough information about the past 
of a system to give us complete information as to its future. 
Nevertheless, as in the case of all ensemblesof time series, the 
theory of the amount of information which we have here developed 
is applicable, and consequently the theory of entropy. Since 
however we now are dealing with time series with the mixing pro- 
perly even when our dataareas completeas they can be, we find 
that our system has no absolute potential barriers, and that in the 
course of time, any state of the system can and will transform 
itself into any other state. However, the probability of this 
depends in the long run on the relative probability or measure 
of the two states. This turns out to be especially high for states 
which can be transformed into themselves by a large number 
of transformations, for states which, in the language of the 
quantum-theorist, have a high internal resonance, or a high 


quantum degeneracy. ‘The benzene ring is an example of this, 


since the two states are equivalent. This suggests that ina 


an wr 
} fj and | | 
Y/ SF 


system inwhich various building-blocks may combine themselves 
intimately in various ways, as when a mixture of amino-acids 
organizes itself into protein chains, a situation where many of 
these chains are alike, and go through a stage of close associa- 
tion with one another, may be more stable than one in which 
they are different. It was suggested by Haldane, in a tentative 
manner, that this may be the way in which genes and viruses 
reproduce themselves ; and although he has not asserted this 
Suggestion of his with anything like finality, I see no cause not to 
retain it as atentalive hypothesis. As Haldane himself has 
pointed out, as no single particle in quantum theory has a per- 
fectly, sharp individuality, it is not possible 1 aay in such a case 
with more than fragmentary accuracy, which of the two exan- 
ples of a gene that has reproduced itself in this manner is the 
master pattern, and which is the copy- 
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This same phenomenon of resonance is known to be very 
frequently represented in living matter. Szent-Gyérgi has 
suggested its importance in the construction of muscles. Subs- 
tances with high resonance very generally have an abnormal 
capacity for storing both energy and information, and such a 
storage certainly occurs in muscle contraction. 

Again, the same phenomena that are concerned in reproduc- 
lion probably have something to do with the extraordinary 
specificity of the chemical substances found in a living organism, 
not only from species to species, but even within the individuals 


ofa spevies. Such considerations may be very important in 
immunology. 


CHAPTER IV 
FEED-BACK AND OSCILLATION 


A palient comes into a neurological clinic. He is not para- 
lyzed, and he can move his legs when he receives the order. 
Nevertheless, he suffers under a severe disability. He walks 
witha peculiar uncertain gait, with eyes downcast, on the ground 
and on his legs. He starts each step with a kick, throwing each 
leg in succession in front of him. If blindfolded, he cannot 
stand up, and totters to the ground. Whal is the matter with 
him ? 

Another patient comes in. While he sits at rest in his chair, 
there seems to be nothing wrong with him. However, offer him 
a cigarette, and he will swing his hand past it in trying to 
This will be followed by an equally futile swing in 
the other direction, and this by still a third swing back, until 
hing but a futile and violent oscillation. 
er, and he will empty itin these swings 
ng it to his mouth. What is the matter 


pick it up. 


his motion becomes not 
Give him a glass of wat 
before he is able to bri 
with him ? 


Both of these patients are suffering from one form or another 


of what is known as atavia. Their muscles are strong and 
healthy enough, but they are unable to organize their actions. 
The first patient suffers from tabes dorsalis. The part of the 
spinal cord which ordinarily receives sensations has been da- 
maged or destroyed by the late sequelae of syphilis. The 
incoming messages are blunted, if they have not totally disap- 
peared. The receptors in the joints and tendons and muscles 
and the soles of his feet which ordinarily convey to him the 
position and state of motion of his legs send no messages which 
his central nervous system can pick up and transmit, and for 
information concerning his posture he is obliged to trust to his 
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eyes and the balancing organs of his inner ear. In the jargon 
of the physiologist, he has lost an important part of his proprio- 
ceptive or kinaesthetic sense. : 

The second patient has lost none 


of his proprioceptive sense. 
His injury is elsewhere 


» in the cerebellum, and he is suffering 
from what is known asa cerebellar tremor or purpose tremor. 
It seems likely that the cerebellum has some function of propor- 
tioning the muscular response to the proprioceptive input, and 
if this proportioning is disturbed, a tremor may be one of the 
results. 

We titus see that for effective 
not only essential that we poss 
performance of these effectors 
the central nervous system, and 
itors be properly combined with 
in from the sense organs to pr 
output to the effectors. 
in mechanical systems. 


action on the outer world, it is 
ess good effectors, but that the 
be properly monitored back to 
that the readings of these mon- 
the other information coming 
oduce a properly proportioned 
Something quite similar is the case 
Let us consider a signal tower on a 


controls a number of levers which 
turn the semaphore signals on or off, and which regulate the 
setting of the switches. However, it does not do for him to 
assume blindly that the Signals and the Switches have followed 
hisorders. It may be that the Switches have frozen fast, or that 
the weight of a load of snow has bent the signal arms, and that 
whal he has Supposed to be the actual State of the switches and 
the signals — his effectors — does not correspond to the orders 
he has given. To avoid the dangers inherent in this contin- 
gency, every effector, switch i 


>» Which conveys 
actual states and performance, 
valent of the repeating of orders navy, according to a 
code by which every subordinate upon the reception of an 


order, must repeat it back to his Superior, to Show that he has 
heard and understood it. It is on Such repeated orders that the 
signalman must act. 

Notice that in this system there is a human link in the chain 
of the transmission and return of information San What we 
shall from now on eall the chain of feed-back. It is true that 
the signalman is not altogether a free agent ; that his switches 


This is 
in the 


the mechanical equi- 
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and signals are interlocked, either mechanically or electrically, 
and that he is not free to choose some of the more disastrous 
combinations., There are however feed-back chains in which 
no human element intervenes. The ordinary thermostat by 
which we regulate the heating of a house is one of these. There 
is a setting for the desired room temperature ; and if the actual 
temperature of the house is below this, an apparatus is actuated 
which opens the damper, or increases the flow of fuel oil, and 
brings the temperature of the house up to the desired level. If 
on the other hand the temperature of the house exceeds the 
desired level, the dampers are turned off, or the flow ef tuel oil 
is slackened or interrupted. [n this way the temperature of the 
house is kept approximately at a steady level. Note that the 
constancy of this level depends on the good design of the thermo- . 
stat, and that a badly designed thermostat may send the tem- 
perature of the house into violent oscillations not unlike the 
motions of the man suffering from cerebellar tremor. 

Another example of a purely mechanical feed-back system 
— the one originally treated by Clerk Maxwell — is that of 
the governor of a steam-engine, which serves to regulate its 
velocity under varying conditions of load. In the original form 
designed by Walt, it consists of two balls attached to pendulum 
rods and swinging on opposite sides of a rotating shaft. They 
are kept down by their own weight or by a spring, and they are 
Swung upwards by a centrifugal action dependent on the angular 
velocity of the shaft. They thus assume a compromise posilion 
likewise dependent on the angular velocity. This position is 
transmitted by other rods to a collar about the shaft, which 
actuates a member which serves to open the intake valves of 
the cylinder when the engine slows down and the balls fall, and 
to close them when the engine speeds up and the balls rise. 
Notice that the feed-back tends to oppose what the system is 
already doing, and is thus negative. fe. 

We have thus examples of negative feed-hacks uel stabilize 
temperature and negative feed -backs to stabilize velocily. There 
are also negative feed-backs to stabilize position, as in the case 
of the steering engines of a ship, which are actuated by the 
angular difference between the position of the wheel and the 
position of the rudder, and always act so as to bring the position 
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of the rudder into accord with that of the wheel. The feed-back 
of voluntary activity is of this nature. We do not will the 
motions of certain muscles, and indeed we generally do not know 
which muscles are to be moved to accomplish a given task ; we 
will, say, to pick up a cigarette. Our motion is regulated by 
some measure of the amount by which it has not yet been 
accomplished. 

The information fed back to the control center tends to oppose 
the departure of the controlled fro 
but it may depend in widely 
The siniplest control systems 


m the controlling quantity, 
different ways on this departure. 
are linear : the output of the effec- 
tor is a linear expression in the input, and when we add inputs, 

we also add outputs. The oulput is read by some apparatus 
. equally linear. This reading is simply subtracted from the input- 

We wish to give a Precise theory of the performance of such a 


Piece of apparatus ; and, in particular, of its defective behavior 


and its breaking into oscillation when it is mishandled or over- 
loaded. 


cular in the last chapter, Here, too, in the rest of the present 
chapler, we are dealing pre hose matters for which 
in pelo mathematics is the 4ppropriate language, and 
we can only avoid it by long Periphrases which are scarcely 
intelligible to the layman, and which are Only intelligible to the 
Beye ec coeinedrsith mathematical Symbolism by virtue of his 
ability to translate them into this Symbolism, 
mise we can make is to supplement the sy, 
verbal explanation. 

Let /(Q) bea function of the time ¢ 
nity to infinity : that is, let /(t) be 
rical value for each time #. Atan 
accessible to us when s is less th 
sis greaterthan?. There are pi 
mechanical, which delay their input by a fixed time a ped 
yield us, for an input /(é), an output fitz), RHE ‘rnc 
delay. 


We) may combine: several Dietetor *Pparatus of this kind 


The best compro- 
mbolism by an ample 


where { runs from minus infi- 
a quantity 


: assuming a nume- 
y time ¢, th 


€ quantities /(s) are 
an or qual to t, but not when 
ces of “PPparatus, electrical and 
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yielding us outputs f(t-t;), f(é-*2)s--+» f(t-tn). Wecan multiply each 
of these outputs by fixed quantities, positive or negative. For 
example, we may use a potentiometer to multiply a voltage by 
a fixed positive number less than one, and it is not too difficult 
to devise automatic balancing devices and amplifiers to multiply 
a voltage by quantities which are negative, or are greater than - 
one. Itis also not difficult to construct simple wiring diagrams 
of circuits by which we can add voltages continuously, and with 
the aid of these, we may oblain an output: 


n 
(4.01) Sdn f(t— 7n)- 
1 
By increasing the number of delays tn, and suitably adjusting 
the coefficient dn, we may approximate as closely as we wish 
to an output of the form: 


(4.02) ve ariGye ce 


0 


it is important to realize that the fact that 


In this expression, 
and not from — © to o, is 


we are integrating from 0 to ©, 
essential. Otherwise we could use various practical devices to 


Operate on this result, and to obtain f(é + s), where oc is posi- 
tive. This however involves the knowledge of the future of 
f(t); and f(t) may be a quantity, like the coordinates of a street- 
car which may turn off one way or the other at a switch, which 
is not determined by its past. Whena physical process seems to 


yield us an operator 


(4.03) hs * a(t) flt—2) at, 


—7« 


where a(t) does not effectively vanish for negative values of J, 
it means that we have no longer a true operator on f(t), deter- 
mined uniquely by its past. There are physical cases where 
this may occur. For example, a dynamical system ae 
input may go into permanent oscillation, or even oscillation 
building up to infinity, with an undetermined amplitude. In 
such a case the future of the system is nol determined by the 
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past, and we may in appearance find a formalism which suggests 
an operator dependent on the future. 
The operation by which we obtain (4.02) from /(é) has two 


important further properties : (1) it is independent of a shift of 
the origin of time ; and (2) itis linear. 
‘pressed by the statement that if 


(4.04) g(t) = y "a i — 2 de, 


0 


The first property is ex- 


then +. 


(4.05) g(t +0) = a * als) ft-+o—-) de. 


0 


The second property is expressed by the slatement that if 


(4.06) gt) =AF(O+B fd), 
then 


(4.07) f * as) g(t--2) de =A tf als) file —s) de 
0 


0 


+B f” a(s) ft—x) dz. 
e 


lt may be shown that in an appropriate sense, every operator 
on the past of {(t) which is linear and is invariant under a shift 
of the origin of time is either of the form (4.02), or is a limit of a 
sequence of operators of that form. For example, f‘() is the 


result of an operator with these properties when applied to 
(i), and 


(0s) FO= 5G f 2a(=) pu ay 


0 
where 
1 (O<X <1); 
(4.09) ahpas p— 1 ey, 
0 2<X). 


* i Z 
As we have seen before, the functions e2 are a Set of func tions 
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{@® which are particularly important from the point of view of 
Operator (4.02), since 
(4.10) 9 eZ{t—z) = e2t , e—Z, 
and the delay operator becomes merely a multiplier dependent 
on Z. Thus (4.02) becomes: 


(4.11) en [~ a(x) e— dz, 


0 
and is also a multiplication operator dependent on Z only. The 
expression 


(4.12) : ” aGyetede = A(Z) 


0 


is said to be the representation of the operator (4.02) as a function of 
frequency. If Z is taken as the complex quantity, x + iy, where 
x and y are real, this becomes. 


(4.13) F a(z) e—@ els de, 
0 


so that by the well-known Schwarz inequality concerning inte- 
grals, if y > 0 and 


(4.14) *| als) [2de <0, 
‘i 
we have: 
20 Pace 1/2 
(4.15) | A(w+ iy) | <j f | a(x) |? oo ena: af 
0 


a lef a(s) | ds - 
0 


This means that A (w +/7y) isa bounded holomorphic func- 
tion of a complex variable in every half-plane x De > 0, and 
that the function A(iy) represents in a certain very definite sense 
the boundary values of such a function. 

Let us put: 


(4.16) utiv=A(e+iy), 
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where u and v are real. The x + iy will be determined as a 
function (not necessarily single-valued) of u + iv. This function 
will be analytic, though meromorphic, except at the points 
u + iv corresponding to points z = a+ iv where dA(z)/az = 0. 


The boundary 2 = 0 will go into the curve with the parametric 
equation : 


(4.17) u+iv=Azx (x real). 


This new curve may intersect itself any number of times. In 
general, however, it will devide the plane into two regions. Let 
us consider the curve (4.17) traced in the direction in which 2, 
goes from — -o tos. Then if we depart from (4.17) to the 
right, and follow a continuous course not again cutling (4-17) 
we may arrive at certain points. The points which are neither 
in this set nor on (4.17) we shall call exlerior points. The part 
of the curve (4.17) which contains limit points of the exterior 
points we shall call the effective boundary. All other points 
will be termed interior points. Thus in the following diagram, 
with the boundary drawn in the sense of the arrow, the interior 
points are shaded and the effective boundary is drawn heavily. 


Yj/j 
The condition that A be bounde 


din any right half-plane will 
then tell us that the point at infinity bak pi be es interior 
point. It may bea boundary point, although Te So outain 
very definite restrictions on the } 


Character of the t un- 
dary point it may be. These concern ype of bo 


F i ; P the « thickness » of the 

set of interior points reaching out to infinity, 
Now we come to the problem of the mathemati ressi 
of the problem of linear feed-back. [¢ ical expression 


t the control 1 
sate i: ow chart 
= Flo (hes Witing A1ABrEM ies OF SUCHEN System be as follows: 


Fic. 1, 
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Subtractor 


Input _————————— eee 


Motor 
operator. A. 


Multiplier 
operator A 


A 


Fic. 2. 


Here the input of the motor is Y, which is the difference between 
the original input X, and the output of the multiplier, which 
multiplies the power output AY of the motor by the factor a. 


Thus: 
(4.18) YA aK ANY 
and 
aati 
(218) [ Eee 


so that the motor output is 


ret 
(4.20) AS Merten” 


d by the whole feed-back mechanism is 
pill be infinite when and only when 
17) for this new operator will be 


The operator produce 
then A((1-+ AA). This t 


A= —1/a. The diagram (4. 
: : A@)_. 
(4.21) u+i0= T7A@)’ 


and x will be an interior point of this when and only when 1h 
is an interior point of (4.17). 

In this case, a feed-back with 
produce something catastrophic. 


a multiplier 4 will certainly 
and as a matter of fact the 
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catastrophe will be that the system will gointo unrestrained and 
increasing oscillation. If on the other hand the point —1,A is 
an exterior point, it may be shown that there will be no difficulty, 
and the feed back is stable. If —1/2 is on the effective boun- 
dary, a more elaborate discussion is necessary. Under most 
circumstances, the system may go into an oscillation of an 
amplitude which does not increase. i 
It is perhaps worth considering several operators A and the | 
ranges of feed-back which are admissible underthem. We shall 
not only consider Operations (4.02), but also their limits, 
assumiitg that the same argument will apply to these. 
If the operator A corresponds to the differential operator, 
A(z)= =z, as y goes trom — o to ox, A(y) does the same, and 
the interior points are the points interior to the right half-plane. 


The point —1/), is always an exterior point, and any amount of 
feed-back is possible. If 


5 1 
4.22) A(Z) = 
( (2) ais? 
the curve (4.17) is 
(4.23 tip 1 
) roel UE hig 
or 
1 " 
4,24 a eee _ ky 
( eT agi ae or 
which we may write : 
(4.25) 


ur + v2 = y, 
This is a circle with radius 1/ 
cribed in the clockwise Sense, and the in 
which we should ordinarily consider int 
the admissible feed-back is unlimited 
the circle. The a(z) corresponding 


2, and center at (1/2,0). It is des- 


terior Points are those 
€rior. In this case too 


78S —1/2 is always outside 
© this operator ig 


(4.26) a(t) = eth), 
Again, let 
oA 4 eo 1 _ 
(4.27) A(z) = (2) F 


Then (4.17) is 


1 2 1 . 
(4.28) u+iv =( a ) ~G~kiy 
( ) l+tkiy CE aya 3 
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and 
jee ati —ky 


(4.29) is ae 
° a + ky)? , (a + ky?)? 2. 


This yields : 


(4.30) eee. 
liad (1+ ky)?” 
or 
(4.31 — (aa 
) y (u2+ py2k 
Then 
(4.32 een: ( ase 
) u=(u? + v’) (1 Tue poe 


v 


= (u2 ve 
(u? + v*) ie +0) Oy 


In polar coordinates, if u = ¢ Cos % 0 = ep sin 9, this becomes 


é i 1 cos? 
4.33 _ 4 Se pe 
( ) e COS@ e rl e 4 rime 
or 

430 cose _ Z 

(4.34) ‘——5 =tz 


That is, 

(4.35) pi/2 = — sin = ‘ pl/2 = cos = : 
o equations represent only one 
at the origin and cusp pointing 
will contain no point of 
the admis- 


Itcan be shown that these tw 


curve, a cardioid with vertex 
The interior of this curve 
and as in the previous case, 
e the operator a(Z) is : 


to the right. 
the negative real axis, 
sible amplification is unlimited. Her 


(4.36) a(t) =t etik/ ke. 
Let 
4.37 : : 
oS a =(sq00) 
tcase. Then 


Let ¢ and ¢ be defined as in the las 
1 


4.38 4/3 cos 2: i pt8 si nt 
(4.38) et cos g + iet® sin = Ty iy * 
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As in the first case, this will give us: 
(4.39) 62/3 cos? = + »2/3 sin? = el/3 cos ze 
That is, 


(4.40) eB = cos =, 


which is a curve of the following shape: 


ol 


Fic: 3, 
The shaded. region represents the interior points. All feed-back 
with coefficient exceeding 1/2 is impossible. The corresponding 
a(t) is: 
(4.41) a(t) =f eth (2 ka), 
Finally, let our Operator corres 
delay of T units of time. Then 
(4.42) 
Then 
(4.43) u+ iv = e—Tiy 
The curve (4.17) will be the unit «i 
described in a clockwise sense aot about ee y ieie 
velocity. The inside of this curve eee witht ue 


will b pao ee i- 

nary sense, and the limit of feed-back Se ede in the ordi 
There is one very interesting conclusion. v gee 1. ; 

€ drawn from 


this. It is possible to compensate fe 
Tee rator 11 + kz) by 


or the 

an arbitrarily heavy feed-back, which wi} give y ‘A A) 
San A((1 +2 

duency range as we 


as near to 1 as we wish for as large ag fre 
for three successive 


Ponding to A be a’ simple 


A(z) = eT, 


fos Ty — isin T y. 


wish. It is thus possible to compensate 
operators of this sort by three — one 


n . 
IWOUE SS successive 
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feed-backs. It isnot, however, possible to compensate as closely 
Se wish for an operator 1/(1 + kz)’, which is the resultant 
of the composition of three operators 1/(1 + kz) in cascade, 
bya single feed-back. The operator 1/(1 + kz)® may also be 
written: 

ide 1 


(4.44) : 
and may be regarded as the limit of the additive composition 
of three operators with first-degree denominators. It thus 
appears thal a sum of different operators, each of which may 
be compensated as well as we wish by a single feed-back, can- 
not itself be so compensated. 

In the important book of McColl, we have an example of a 
complicated system which can be stabilized by two feed-backs, 
but not by one. It concerns the steering of a ship by a gyro- 
compass. ‘he angle between the course set by the quartermaster 
and that shown by the compass expresses itself in the turning 
of the rudder, which in view of the headway of the ship, produces 
a turning moment which serves to change the course of the ship 
as to decrease the difference between the set 
course and the actual course. If this is done by a direct opening 
of the valves of one sleering-engine and closing of the valves 
of the other in such a way that the turning-velocity of the rudder 
is proportional to the deviation of the ship from this course, let 
us note that the angular position of the rudder is roughly 
proportional to the turning-moment of the ship and thus to its 
angular acceleration. Hence the amount of turning of the ship 
is proportional with a negative factor to the third derivative of 
the deviation from the course, and the operation which we have 
to stabilize by the feed-back from the gyro-compass is kz3, 
where k is positive. We thus get for our curve (4.17) 


mm such a way 


(4.45) utiv=—kiy’, 


he left-half plane is the int 
stabilize the system. 

In this account, we have slightly oversimplified the steering 
Actually there is a certain amount of friction, and 
rning the ship does not determine the acceleration. 


andast erior region, no servo-mecha- 


nism whatever will 


problem. 
the force tu 


126 CYBERNETICS 


Instead, if 6 is the angular position of the ship and ¢ that of 
the rudder with respect to the ship, we have : 


d?0 , de 

(4.46) ar tat — eae 
and 

(4.47) u+ iv==-—kiy®— ky?. 
This curve may be written : 

(4.48) yp? = — kus, 
which still cannot he stabilized by any feed-back. As y' Bes 
from -= cto o%, u goes from o to — oo, and the inside of the 


curve is to the left. 


If on the other hand the position of the rudder is proportional 
to the deviation of the course, the operator to be stabilized by 
feed-back is k,z? + kz, and (4.17) becomes: 


(4.49) u+ivs—ky?+ k,iy 


This curve may be written 
(4.50) 


v? =—- kyu, 

but in this case, as Y goes from — c to ce, so does v, and the 
curve is described from Y =— eto y = oo. In this case, the 
outside of the curye is to the left, and unlimited amount of 
amplification is possible. 


To achieve this we may employ another Stage of feed-back. 


If we regulate the position of the valves of the steering engine, 
not by the discrepancy between the actual and the desired 
course, but by the difference bet 


ey ween this quantity and the 
angular position of the rudder, we shall keep the angular posi- 


tion of the rudder as nearly Proportional to the ship’s devia- 


tion from true course as we wish, if we allow a large enough 
feed-back — that is, if we Open the y. 


alves wide enough. This 

double feed-back system of control is in fact the one usually 

adopted for the automatic Steering of ships by means of the 
gyro-compass. 

In the human body, the motion ofa hand OF afingepinvolves 

a system with a large number of joints, The out ae an addi- 

tive vectorial combination of the outputs of all ilies ‘phte. Wie 

have seen that in general,a complex additive ee like this 

cannot be stabilized by a single feed-back. Correspondingly the 
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voluntary feed-back by which we regulate the performance of a 
task through the observation of the amount by which it is not 
yet accomplished needs the backing up of other feed-backs. 
These we call postural feed-backs, and they are associated with 
the general maintenance of tone of the muscular system. It is 
the voluntary feed-back which shows a tendency to break down 
or become deranged in cases of cerebellar injury, for the ensuing 
tremor does not appear unless the patient tries to perform a 
voluntary tasx. This purpose tremor, in which the patient can- 
not pick up a glass of water without upsetting it, is very 
different in nature from the tremor of Parkinsonianism or 
paralysis agitans, which appears in its most typical form when 
the patient is at rest, and indeed often seems to be greatly 
mitigated when he attempts to perform a specific task. There 
are surgeons wilh Parkinsonianism who manage to operate quite 
efficiently. Parkinsonianism is known not to have its origin in 
a diseased condition of the cerebellum, but to be associated 
with a pathological focus somewhere in the brain-stem. It is 
only one of the diseases of the postural feed-backs, and many of 
these must have their origin in defects of parts of the nervous 
system situated very differently. One of the great tasks of 
Physiological cybernetics is to disentangle and isolate loci of 
the different parts of this complex of voluntary and postural 
feed backs. Examples of component reflexes of this sort are 
the scratch and the walking reflex. 

When feed-back is possible and stable, its advantage, as we 
have already said, is to make performance less dependent on 
the load. Let us consider that the load changes the characteris- 
tic A by dA. The fractional change will be dA/A. If the ope- 
rator after feed-back is ; 


(4.51) B= | 


we shall have : 


(Ci) ee 


Thus feed-back serves to diminish the dependence of the system 
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on the characteristic of the motor, and serves to stabilize it, 
for all frequencies for which 


(4.53) | e - 


sails 


This is to say thal the entire boundary between interior and 
exterior points must lie inside the circle of radius C about the 
point — C. This will not even be true in the first of the cases 
we have discussed. The effect of a heavy negative feed-back, if 
it is at all stable, will be to increase the stability of the system 
for low frequencies, but generally at the expense of its stability 
for some high frequencies. There are many cases in which 
even this degree of stabilization is advantageous. ; 
A very important question which arises in connection with 
oscillations due to an excessive amount of feed-back is that of 
the frequency of incipient oscillation. ‘This is determined by 
the value of y in the iy corresponding to the point of the 
boundary of the inside and outside regions of (4.17) lying 


furthest to the left on the negative u-axis The quantity y is of 
course of the nature of a frequency. 


We have now come to the end 


of an elementary discussion of 
linear oscillations, studied from 


the point of view of feed-back. 
A linear oscillating System has certain very special properties 
which characterize its oscillations. Qne is that when it oscil- 
eee ae alwayaycant and very generally — in the absence of 
independent simultaneous oscillations — does oscillate in the 
form 
(4.54) A sin (Bt C) ent, 
The existence of a periodic non-sinus, 
a suggestion at least that the variab] 
the system is nol linear. In some 
system may be rendered linear 
the independent variable. 

Another very significant difference between Hee haa esis 
linear oscillations is that in the first, the amplitude of oscillation 
is completely independent of the frequency . mrilbhdatedates 
there is generally only one amplitude 


: Week Ostial diccretercet 
of amplitudes for which the system wi1 °scillate at a given 


Oidal oscillation is always 
© observed is one in which 
“ases, but in very few, the 
again” by a new choice of 
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frequency, as well as a discrete set of frequencies for which the 
syslem will oscillate. This is well illustrated by the study of 
what happens in an organ pipe. There are two theories of the 
organ pipe — a cruder linear theory, and a more precise non- 
linear theory. In the first, the organ pipe is treated as a conser- 
vative system. No question is asked about how the pipe came to 
oscillate, and the level of oscillation is completely indeterminate. 
In the second theory, the oscillation of the organ pipe is consid- 
ered as dissipating energy, and this energy is considered to 
have its origin in the stream of air across the lip of the pipe. 
There is indeed a theoretical steady-state flow of air across the 
lip of the pipe which does not interchange any energy with any 
of the modes of oscillation of the pipe, but for certain velocities 
of air-flow, this steady-state condition is unstable. The slightest 
chance deviation from it will introduce an energy inpul into one 
or more of the natural modes of linear oscillation of the pipe ; 
and up to acertain point, this motion will actually increase the 
coupling of the proper modes of oscillation of the pipe with the 
energy input. The rate of energy input and the rate of energy 
output by thermal dissipation and otherwise have different laws 
of growth, but to arrive at a steady state of oscillation, these 
two quantities must be identical. Thus the level of the non- 
linear oscillation is determined just as definitely as its frequency. 

The case we have examined is an example of what is known as 
a relaxation oscillation: a case, that is, where a system of 
equations invariant under a translation in time, leads to a solu- 
tion periodic — or corresponding to some generalized notion 
of periodicity — in time, and determinate in amplitude and 
frequency, but not in phase. In the case we have discussed, 
the frequency of oscillation of the system is close to that of some 
loosely coupled, nearly linear part of the system. B. Van der 
Pol, one of the chief authorities on relaxation oscillations, has 
pointed out that this is not always the case, and that there are 
in fact relaxation oscillations where the predominating frequency 
is not near the frequency of linear oscillation of any part of 
the system. An example is given by a stream of gas flowing 
into a chamber open to the air and in which a pilot light is 
burning : when the concentration of gas in the air reaches a 
certain critical value, the system is ready to explode under 
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ignition by the pilot light, and the time it takes for this to happep 
depends only on the rate of flow of the coal gas, the rate al 
which air seeps in and the products of combustion seep out, 
and the percentage composition of an explosive mixture of coal 
gas and air. 

In general, non-linear systems of equations are hard to solve. 
There is however a specially tractable case, in which the system 
differs only slightly from a linear system, and the terms which 
distinguish it change so slowly that they may be considered 
substantially constant over a period of oscillation. In this case, 
we may study the non-linear system as if it were a linear system 
with slowly varying parameters. Systems which may be stu- 


died this way are said to be perturbed secularly, and the theory 


of secularly perturbed systems plays a most important role in 
gravitational astronomy. 


It is quite possible that som 
be treated somewhat rough 
systems. We can see quile 


© of the physiological tremors may 
ly as secularly perturbed linear 


clearly in such a system why the 
steady-state amplitude level may be just as determinate as the 
frequency. Let one element in Such a system be an amplifier 
whose gain decreases as Some long-time average of the input of 
such a system increases, Then as the oscillation of the system 
builds up, the gain may be cut down, until a stale of equili- 
brium is reached. 

Non-linear systems of relaxation Oscillations haye been stu- 
diedin some cases by methods developed by Hill and Poincaré !- 
The classical cases for the Study of such oscillations are those in 
which the equations of the Systems are of . differential nalnnes 
especially where these differentia] €quations are of low order: 
There is not, as far as I know, any comparable adequate study 
of the corresponding integral €quations, when bse system 
depends for its future behavior On its entire past behavior. 
However, it is not hard to sketch out the eatiees h a theory 
should take, especially when we are only looking on : riodic 
solutions. In this case the slight Modification af id or ah a 
of the equation should lead to g slight, ~ ones 


linear, modification of the equations of motion. For exan ple, let 
. nple, 


1. Poincaré, H., Les Méthodes Nouvelles dais la 


Mécanique Céleste. 
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Op} f(t)! be a function of t which resulls from a non-linear 
operation on f(t), and which is affected by a translation. Then 
the variation of Op | f() |, 8 Op } /() | corresponding to a varia- 
tional change 3 (2) in f(t) and a known change in the dynamics 
of the system, is linear but not homogeneous in 3/(), though 
not in /(é). If we now know a solution /(2) of 


(4.55) Op! fH, =9 
and we change the dynamics of the system, we obtain a linear 
non-homogeneous equation for 8 /(). If 


(4.56) fit) = & anein, 
and /(1) +8 f(1) is also periodic, being of the form : 


20 


(4.57) f(t) + aft) = J Cau + Saneintr anne; 


then 
(1.58) f(t) = Nba, etme +N ayeiom in bat. 
SS = 


The linear equations for 8 /(n) will have all coeflicients develop- 
able into series in e, since (1) can itself be developed in this 
form. We shall thus obtain an infinite system of linear non- 
homogeneous equations in 8an+an in § a,andaand this system of 
equations may be solvable by the methods of Hill. In this case 
it is at least conceivable that by starting with a linear equation 
{non-homogeneous) and gradually shifting the constraints, we 
may arrive at a solution of a very general type of non-linear 
problem in relaxation oscillations. This work, however, lies in 
the future. 

To a certain extent, the feed-back systems of control discussed 
in this chapter and the compensation systems discussed in the 
last are competitors. They both serve to bring the complicated 
input-output relations of an effector into a form approaching a 
simple proportionality. The feed-back system, as we have seen, 
does more than this, and has a performance relatively inde- 
pendent of the characteristic and changes of characteristic of the 
effector used. The relative usefulness of the two methods of 
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control thus depends on the constancy of the characteristic of 
the effector. It is natural to suppose that cases arise in which it 
is advantageous to combine the two methods. ‘There are various 


ways of doing this. One of the most simple is that illustrated 
in the following diagram: 


Subtractor 


Effector 


Feed-back 
take-off 


Out put 


Fic. 4. 


In this, the entire feed-back system may be regarded as a larger 
effector, and no new point arises, 


except that the compensator 
must be arranged to compensate w 


ny hat is in some sense the ave- 
rage characteristic of the feed-bac 


e k system. Another type of 
arrangement is the following : 


> 
Subtractor Out put: 


Here the compensator and effector are Combined into one 
larger effector. This change will in general. alter Pe (0) ef 
mum feed-back admissible, and it js nol eae fv 
can ordinarily be made to increase that level to 4 See 10W ; 
extent. On the other hand, for the same f pet 


€ed-bac 

most definitely improve the performance SU the aati If, for 

example, the effector has an essentially lagging elioractes 5 ky 
a ris 2 
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the compensator will be an anticipator or predictor, designed for 
its statistical ensemble of inputs. Our feed-back, which we may 
call an anlicipatory feed-back, will tend to hurry up the action 
of the effector mechanism. 

Feed-backs of this general type are certainly found in human 
and animal reflexes. When we go duck-shooting, the error 
which we try to minimize is not that between the position of the 
gun and the actual position of the larget, but that between the 
Position of the gun and the anticipated position of the target. 
Any system of anti-aircraft fire control must meet the same pro- 
hlem. The conditions of stability and effectiveness of anticipa- 
tory feed-backs need a more thorough discussion than they have 
yet received. 

Another interesting variant of feed-back systems is found in 
the way in which we steer a car on an icy road. Our entire 
Conduct of driving depends on a knowledge of the slipperiness 
of the road-surface : that is, on a knowledge of the performance 
characteristics of the system car-road. If we wait to find this 
out by the ordinary performance ofthe system, we shall discover 
ourselves in a skid before we know it. We thus give to the 
steering-wheel a succession of small fast impulses, not enough 
to throw the car into a major skid, but quite enough to report to 
our kinaesthetic sense whether the car is in danger of skidding, 
and we regulate our method of steering accordingly. 

This method of control, which we may call control by infor- 
mative feed-back, is not difficult to schematize into a mechanical 
form,and may well be worth while employing in practice. We 
havea compensator for our effector, and this compensator has 
a characteristic which may be varied from outside. We super- 
impose on the incoming message a weak high-frequency input, 
and take off the output of the effector a partial output of the 
Same high frequency, separated from the rest ion the output by 
an appropriate filter. We explore the amplitude-phase rela- 
tions of the high-frequency output to the input in order to 
obtain the performance characteristics of the effector. On the 
basis of this, we Modify in the appropriate sense the character- 
istics of the compensator. The flow-chart of the system is 
much as in the following diagram : 
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Original 
input 


Adder 


Control determining sani 
isti omparator 
compensator | characteristic Mk offe 
characteristics 


Fic. 6. 


The advantages of this t 


ype of feed-back are that the compen- 
sator may be adjusted to 


give Stability for every type of cons- 
tant load ; and that, if the characteristic of the load changes 
slowly enough, in what we have called a secular manner, in 


comparison with the changes of the original input, and if the 
reading of the load condition j 
tendency 


1S again on the lemperature ; but 


Of course, this informati 
the characteristics of the load at high fre 
as, or give a good indication its ch ae fre- 

: fae bce ara w [re 
quencies. This will often be the c  emesgi 
and hence of the effector, 


Sa relatj er of 
variable parameters, vely small numb 


This informative feed-back and t ; 
of feed-back with compensators are eae =i eae po 
is a very complicated theory, ang a theory as trae ne ctly 
studied. The whole field is undergoing a ee yet imper 7 y 
ment. It deserves much more attention in a oe = ; 
Before we end this chapter, we mus not jateiga hin a 
portant physiological application Of the principle ie ae 4 
A great group of Palace witieh Some sort of fed ing not 
only exemplified in physiological Phenomena, a Soe. 


contain 


FEED-BACK AND OSCILLATION 135 


essential for the continuation of life, is found in what is known 
as homeostasis. The conditions under which life, especially heal- 
thy life, can continue in the higher animals, are quite narrow. 
A variation of one half degree centigrade in the body tempera- 
ture is generally a sign of illness, and a permanent variation of 
five degrees is scarcely consistent with life. The osmotic pres- 
sure of the blood and its hydrogen-ion concentration must be 
held within strict limits. The waste products of the body must 
be excreted before they rise to toxic concentrations. Besides all 
these, our leucocytes and our chemical defenses against infection 
must be kept at adequate levels ; our heart rate and blood pres- 
sure must neither be too high nor too low; our sex cycle must 
conform to the racial needs of reproduction ; our calcium 
metabolism must be such as neither to soften our bones nor to 
calcify our tissues ; and so on. In short, our inner economy 
must contain an assembly of thermostats, automatic hydrogen- 
ion-concentration controls, governors, and the like, which would 
be adequate for agreat chemical plant. These are what we know 
Collectively as our homeostatic mechanism. 

Our homeostatic feed-backs have one general difference from 
Our voluntary and our postural feed-backs : they tend to be slower. 
There are very few changes in physiological homeostasis 
— not even cerebral anemia — that produce serious or perma- 
nent damage in a small fraction of a second. Accordingly, the 
nerve fibres reserved for the processes of homeostasis— the sym- 
pathetic and parasympathetic systems — are often non-myeli- 
nated, and are known to have a considerably slower rate of trans- 
mission than the myelinated fibres. The typical effectors of 
homecstasis — smooth muscles and glands — are likewise slow in 
their action compared with striped muscles, the typical effectors 
of voluntary and postural activily. Many of the messages of the 
homeostatic system are carried by non-nervous channels — the 
direct anastomosis of the muscular fibres of the heart, or chemical 
messengers such as the hormones, the carbon dioxide content of 
the blood, ete. ; and except in the case of the heart muscle, these 
too are generally slower modes of transmission than myelinated 
nerve fibres. 

Any complete text-book on cybernetics should contain a 
thorough detailed discussion of homeostatic processes, many 
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individual cases of which have been discussed in the literature 
‘ with some detail?. 
to the subject th 
« homeostatic. pro 


3 4 eres tion 
However, this book is rather an introduce ‘ 

; : ry 
an a compendious treatise, and the theory 


: c ledge 
cesses involves rather too detailed a knowledg 
_ of general physiology to be in place here. 

: ; an 43) Miiness of 
“1. Cannon, W.. The Wisdom of the Body : Henderson, LJ. The Fitness 
the Environment. 


CHAPTER V 


COMPUTING MACHINES = 
AND THE NERVOUS SYSTEM 


Computing machines are essentially machines for recording 
numbers, operating with numbers, and giving the resull in nume- 
ticalform. A very considerable part of their cost, bothin money 
and in the effort of construction, goes to the simple problem of 
recording numbers clearly and accurately. The simplest mode 
of doing this seems to be on a uniform scale, with a pointer of 
Some sort moving over this. If we wish to record a number 
with an accuracy of one part in n, we have to assure that in each 
region of the scale, the pointer assumes the desired posilion 
within this accuracy. That is, for an amount of information 
log, n, we must finish each parl of the movement of the pointer 
With this degree of accuracy, and the cost will be of the form 
An, where A is not too far from a constant. More precisely, 
see if n-j regions are accurately established, the remaining 
region will also be determined accurately, the cost of recording 
an amount of information I will be about 


(5.01) (02a) Ac 


a let us divide this information over two scales, each marked 
o accurately. The cost of recording this information will be 
about : 


(5.02) 2 (5 baal ane 


If the information be divided among N scales, the approximate 
cost will be : 


(5.03) N (ox a 1) A. 
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This will be a minimum when : 
I ieee 
(5.04) ZN 1 = <9 jog 3: 
or if we put : 


(5.05) A 


a log 2=2, 
when 
(5.06) a 


er 
This will occur when, and only when, x = 0, or N= 2. They 
is, N should be as large as Possible to give the lowest cost for 
the storage of information. Let us remember that 2UN ‘must be 
an integer, and that 1 is not a Significant value, as we then have 


an infinite number of Scales each containing no acelin 
The best significant Value for QI/N js 2, in which case we reco! 
our number on a n 


c ivi d 
umber of independent scales, each divide 
Into two equal parts In other Words, we represent our numbers 
in the binary system 


Oe Yet see ft aly un be, 
where every vis either 1 or 0. 

There exist at Present two g 
those, like the Bush different 
analogy machines, where { 
surements on some continuous Scale, so t 
machine is determined by the of Construction of the 
scale; and those, like the ordinary desk adding ang multiplying 
machine, which we call numerical machines, where the datatare 
represented bya set of choices among a number of Contingencies : 
and the accuracy is determined by the sh 


Reker Be arpness with which 
the contingencies are distinguished, the number of Nea 
1. Journal of the Franklin Institute, Various papers. 1930 on. 


reat types of ¢ 
ial analyzer 1 
he data are 


©mputing machines : 
» Which are known as 
represented by mea- 


hat the accuracy of the 
accuracy 
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contingencies presented at every choice, and the number of 
choices given. We see that for highly accurate work, al any 
rate, the numerical machines are preferable, and above all, 
those numerical machines constructed on the binary scale, in 
which the number of alternatives presented at each choice is two. 
Our use of machines on the decimal scale is conditioned merely 
by the historical accident that the scale of ten, based on our 
fingers and thumbs, was already in use when the Hindus made 
the great discovery of the importance of the zero and the advan- 
lage of a positional system of notation. It is worth retaining 
when a large part of the work done with the aid of the machine 
Consists in transcribing on to the machine numbers in the conven- 
tional décimal form, and in taking off the machine numbers which 
must be written in the same conventional form. 

This is in fact the use of the ordinary desk computing machine, 
4s employed in banks, in business offices, and in many statistical 
laboratories. Itis not the way that the larger and more automatic 
machines are best to be employed ; in general, any computing 
machine is used because machine methods are faster than hand 
methods. In any combined use of means of computation, as in 
any combination of chemical reactions, it is the slowest which 
gives the order of magnitude of the time constants of the entire 
System. It is thus advantageous, as far as possible, to remove the 
human element from any elaborate chain of computation, and 
Only to introduce it where it is absolutely unavoidable, at the 
very beginning and the very end. Under these conditions, it 
Pays to have an instrument for the change of the scale of nota- 
tion, to be used initially and finally in the chain of compula- 
tions; and to perform all intermediate processes on the binary 
Scale. 

The ideal computing machine must then have all its data 
inserted at the beginning, and must be as free as possible from 
human interference to the very end. This means that not only 
must the numerical data be inserted alt the beginning, but also 
all the rules for combining them, in the form of instructions 
covering every situation which may arise in the course of the 
computation. Thus the computing machine must be a logical 
machine as well as an arithmetic machine, and must combine 
contingencies in accordance with a systematic algorithm. While 
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there are many algorithms which might be used for era 
contingencies, the simplest of these is known as the — 
logic par excellence, or the Boolean algebra. This algorithm, ike 
the binary arithmetic, is based on the dichotomy, the choice 
between yes and no, the choice between being in a class and 
outside it. The reasons for its superiority to other systems are 
of the same nature as the reasons for the superiority of the 
binary arithmetic over other arilhmeltics. : 
Thus all the data, numerical or logical, put into the machine, 


are in the form ofa set of choices between two alternatives; and 


all the operations on the data take the form of making a set of 


new choices depend on a set of old choices. When I add 
two one-digit numbers, A and B, I obtai 
commencing with 1, if A and B are both 1 
The second digit is 1 if A+B, and is othe 
of numbers of more than one 
complicated rules. 

the decimal, may be 
addition of numbers 
numbers take on the 


na two-digit number 
» and otherwise with 0. 
rwise 0. The addition 
digit follows similar but more 
Multiplication in the binary system, as in 
reduced to the multiplication table and the 
; and the rules for multiplication for binary 
peculiarly simple form given by the table : 

| 

0/0 
ite 


(5.08) 0 
1 
Thus multiplication is si 
new digits when old digits are given. 

On the logical side, if 0 is a negatiy. 


every operator can be derived fro 
transforms 1 into 0 and O into 1 


mply a method to determine a set of 


e and La positive decision, 
m three : negation, which 
; logical addition, with the table 


®|0 1 
LAS 
(5.09) O lon 
ie i 


and logical multiplication, 


with the same table as the numerical 
multiplication of the (1,0) 


System, namely : 
@]90 1 
a 
0/00 
Ijo 1 


. (5.10) 
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That is, every contingency which may arise in the operation of 
the machine simply demands a new set of choices of contingencies 
I and O, depending according to a fixed set of rules on the 
decisions already made. In other words, the structure of the 
machine is that ofa bank of relays, capable each of two conditions, 
say «on», and «off»; while at each stage, the relays assume 
each a position dictated by the positions of some or all the relays 
of the bank at a previous stage of operation. ‘These stages of 
Operation may be definitely « clocked » from some central clock 
or clocks; or the action of each relay may be held up until all 
the relays which should have acted earlier in the process have 
gone through all the sleps called for. 

The relays used in a computing machine may be of very varied 
character. They may be purely mechanical; or they may be 
electro-mechanical, as in the case of a solenoidal relay, in which 
the armature will remain in one of two possible positions of 
equilibrium until an appropriate impulse pulls it to the other 
side. They may be purely electrical systems with two altern- 
ative positions of equilibrium, either in the form of gas-filled 
tubes, or what is much more rapid, in the form of high-vacuum 
tubes. The lwo possible states of a relay system may both be 
Stable in the absence of outside interference, or only one may 
be stable, while the other is transitory. Always in the second 
Case, and generally in the first case, it will be desirable to have 
Special apparatus to retain an impulse which is to act at some 
future lime, and to avoid the clogging up of the system which 
will ensue if one of the relays does nothing but repeat itself 
indefinitely. However, we shall have more to say concerning 
this question of memory later. 

It is a noteworthy fact thal the human and animal nervous 
Systems, which are known to be capable of the work of a 
Computation system, contain elements which are ideally suited 
to act as relays. These elements are the so-called neurons or 
Nerve cells. While they show rather complicated properties 
Under the influence of electrical currents, in their ordinary 
Physiological action they conform very nearly to the « all-or- 
None » principle : that is, they are either at rest ; or when they 
“fire », they go through a series of changes almost independent 
of the nature and intensity of the stimulus. There is first an 
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active phase, transmitted from one end to the other of the neuron 
with a definite velocity, to which there succeeds a refractory 


period, during which the neuron is either incapable of being 
stimulated, or at any rate is not ca 


any normal, physiological process, 
refractory period, the nerve 
stimulated again into activity. 

Thus the nerve may be taken to be 
two states of activity : firing 
neurons which accept their 
sory end-oryans, e 


pable of being stimulated by 
At the end of this effective 
remains inactive, but may be 


a relay with essentially 
and repose. Leaving aside those 
messages from free endings or sen- 
ach neuron has its messages fed into it by 
other neurons at Points of contact known as synapses. For a 
given oulgoing neuron, these vary in number from a very few 
to many hundred. [tis the state of the incoming impulses at 
the various synapses, combined with the antecedent state of the 
outgoing neuron itself, which determines whether it will fire or 
not. Ifit is neither firing nor refractory, and the number of 


incoming Synapses which « fire » within a certain very short 
fusion inter 


val of time exceeds a certain threshold, then the 
neuron will fire after a known, fairly constant synaptic delay. 

This is perhaps an Over-simplification of the picture : the 
« threshold » may not depend simply 
but on their « weight » and their 


on the number of synapses, 
geometrical relations to one 
n inlo which they feed; and 
that there exist synapses of 
€, the so-called « inhibitory synapses », which 
prevent the firing of the outgoing neuron, or 
alse its threshold with respect to stimulation at the 
ordinary Synapses. What is pretty clear, hiGorepers dp vhat secure 
definite combinations of impulses on the incoming neurons 
having synaplic connections with 4 given neuron will cause il 


to fire, while others will not cause it to fire, This is not to say 
that there may not be other, 


of : humoral nature, Produce slow, secular changes 
tending to vary the A ere eget bs 
e 8 Vary that pattern of coming impulses which is ade 
quate for firing, 


a different natur 
either complete] 
al any rate yr: 


A very important function of the nervous system, and, as we 

* . 7% : on a¢ 
have said, a function equally in demand foy computing machines, 
is that of memory, the ability to Preserve the results of past 
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Operations for use in the future. It will be seen that the uses 
of the memory are highly various, and it is improbable that 
any single mechanism can salisfy the demands of all of them. 
There is first the memory which is necessary for the carrying 
out of a current process, such as a multiplication, in which the 
intermediate results are of no value when once the process is 
completed, and in which the operating apparatus should then be 
released for further use. Such a memory should record quickly, 
be read quickly, and be erased quickly. On the other hand, 
there is the memory which is intended to be part of the files, 
the permanent record, of the machine or the brain, and to con- 
tribute to the basis of all its future behavior, at least during a 
Single run of the machine. Let it be remarked parenthetically 
that an important difference between the way in which we use 
the brain and the machine is that the machine is intended for 
Many successive runs, either with no reference to each other, 
or with a minimal, limited reference; and that it can be cleared 


between such runs; while the brainin the course of nature, never 


€ven approximately clears out its past records. Thus the brain, 


under normal circumstances, is not the complete analogue of 
the computing machine, but rather the analogue of a single 
run on such a machine. We shall see later thal this remark 
has a deep significance in psychopathology and in psychiatry. 

To return to the problem of memory, a very satisfactory me- 
thod for constructing a short-time memory is to keep a BeqiTence 
of impulses travelling around a closed circuit, until this circuit 
is cleared by intervention from outside. There is much reason 
to believe that this happens in our brains during the retention of 
impulses which occurs over whal is known as the specious pre- 
Sent. This method has been imitated in several devices which 
have been used in computing machines, or at least suggested 
for such a use. There are two conditions which are desirable 
in such a retentive apparatus: the impulse should be transmit- 
led in a medium in which it is not too difficult to achieve a 


Considerable time-lag ; and before the errors inherent in the 
instrument have blurred it too much, the impulse should he 
reconstructed in a form as sharp as possible. The first condi- 
tion tends to rule out delays produced by the transmission of light, 
5 ric circuits, while it favors the 


OF even « 
r even, in many cases, by elect 
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H 4 j ce ribra- 
use of one form or another of elastic vibrations 3 and such vibra 
> thi i om- 
tions have actually heen employed for this purpose in c ; 
¢ . . . es . i se 
puting machines. If electric circuits are used for delay purposes, 


the delay produced at every stage is rel 


atively short ; or as in 
j if age 1S 
all pieces of linear apparatus, the deformation of the messag 
cumulative, and very 


soon becomes intolerable. ‘To avoid this, 
a second consideration comes into pl 


ay ; we must insert some- 
where in the cycle a rel 


ay which does not serve to repeat the 
form of the incoming message, but rather to trigger off ayn 
message of prescribed form. This is done very easily in the 
nervous system, where indeed all tr 


a trigger phenomenon. In the electrical industry, pieces of 
apparatus for this purpose have long been known, and have been 
used in connection with telegraph circuits. They are known a 
telegraph-lype repeaters. The great difficulty of using them for 
memories of long duration is that they have to function without 
a flaw over an enormous number of consec 
ration. Their success is all the more remarkable : in a piece of 
apparatus designed by Mr. Williams of the University of Man- 
chester, a device of this sort with a unit delay of the order of 
a hundredth of a second has continued in successful operation 


, A 4 at 
for several hours, What makes this more remarkable is tha 
this apparatus was not used merely to preserve a single deci- 
sion, a single « yes » or 


«no », but a matter of thousands 0 
decisions. 


Been 7 : f 
ansmission is more or less of 


ulive cycles of ope- 


Like other forms of apparalus intended to ret 
ber of decisions, this works on 


of the simplest modes of storing information for a relatively 
short time is as the charge on a condenser ; and when this is 
supplemented by a telegraph-type repeater, 
quate method of Storage. To use to 
circuit facilities allached to such a stor: 
to be able to switch su 
condenser to another. 


ain a large num- 
: ~ re 
the scanning principle. O1 


it becomes an ade- 
the best advantage the 
age system, it is desirable 
and very rapidly from one 


The ordinary means of doing this in- 
volve mechanical inertia, and this is never consistent with vers 


high speeds. A much better Way is the use of a large number 
of condensers, in which one plate is either a small piece of 
metal sputtered in to a dielectric, or the imperfectly insula- 
ling surface of the dielectric itself, while one of the connectors 


Ccessively 


COMPUTING MACHINES AND THE NERVOUS SYSTEM 145 


to these condensers is a pencil of cathode rays moyed by the 
condensers and magnets of a sweep-circuit over a course like 
that ofa ploughin a ploughed field. There are various elabora- 
tions of this method, which indeed was employed in a some- 
what different way by the Radio Corporation of America before 
it was used by Mr. Williams. 

These last-named methods for storing information can hold a 
message for quite an appreciable time, if not for a period com- 
parable with a human life-time. For more permanent re- 
cords, there is a wide variety of alternatives among which we 
can choose. Leaving out such bulky, slow, and unerasable 
methods as the use of punched cards and punched tape, we have 
magnetic tape, together with its modern refinements, which have 
largely eliminated the tendency of messages on this material 
to spread ; phosphorescent substances ; and above all, photo- 
graphy. Photography is indeed ideal for the permanence and 
detail of its records, ideal again from the point of view of the 
Shortness of exposure needed to record an observation. It 
Suffers from two grave disadvantages - the time needed for de- 
velopment, which has been reduced to a few seconds, but is 
Still not small enough to make photography available for a 
short-time memory ; and (at present) the fact that a photogra- 
Phic record is not subject to rapid erasure and the rapid im- 
Planting of a new record. The Eastman people have been 
Working on just these problems, which do not seem to be neces- 
Sarily insoluble, and it is possible that by this time they have 


found the answer. 
Very many of the methods of storage of 
Considered have an important physical e 


hey seem to depend on systems with a 
tum degeneracy ; or in other words, with a large number of 


Modes of vibration of the same frequency. This is certainly 
true in the case of ferromagnetism, and is also true in the 
Case of materials with an exceptionally high dielectric cons- 
tant, which are thus especially valuable for use in condensers 
for the storage of information. Phosphorescence as well is a 
Phenomenon associated with a high quantum degeneracy ; and 
the same sort of effect makes its appearance in the photographic 
Process, where many of the substances which act as developers 


information already 
lement in common. 
high degree of quan- 
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seem to have a great deal of internal resonance. ae. 
degeneracy appears to be associated with the ability ed aie 
small causes produce appreciable and stable effects. We “ 
already seen in Chapter II that substances with high ae 
degeneracy appear to be associated with many of the prob e ; 
of metabolism and reproduction. It is probably notan eye 
that here, ina non-living environment, we find them associate 
with a third fundamental property of living matter : the ability 
to receive and organize impulses, and to make them effective 
in the outer world. 

We have seen in the case of photography and similar pro- 
cesses that it is possible to store a message in the form of 


. i re- 
a permanent alteration of certain storage elements. In 
inserting this information int 


these changes to affect the 
One of the simplest way: 
ments which are change 
transmission of message 
in their character due t 
they will transport mes 


© the system, it is necessary to cause 
messages going through the system. 
S to do this is to have as the storage ele- 
d parts which normally assist in the 
Ss, and of such a nature thal the change 
© storage affects the manner in which 
Sages for the entire future. In the ner- 
sort, and it is 


nformation is stored ee 
long periods by changes in the thresholds of hewronas oy wha 


may be regarded as another way of saying the same thing, by 
changes in the permeability of each Synapse to messages. Many 
of us think, in the absence of a better explanation of the 
phenomenon, that the Storage of information in the brain ca 
actually occur in this way. It is conceivable for such a storage 
to take place either by the Opening of new paths or by the clo- 
sure of old ones. ly it is adequately established that 

€ brain after birth. Itis possible, 
napses are formed ; and it iS 
he chief changes of thresholds in the 
are increases. If this is the case, our whole 


attern of Balzac’s Peau de Chagrin, and the very 
process of learning and remembering exhausts our powers of 
learning and remembering, 


until life itself squanders our capital 
stock of power to live, It may well be that this phenomenon? 
does occur. This is a Possible explanation for a sort of senes~ 


memory process 
life is on the p 
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cence. The real phenomenon of senescence, however, is 
much too complicated to be explained in this way alone. 

We have already spoken of the computing machine, and con- 
sequently the brain, as a logical machine. It is by no means 
trivial to consider the light cast on logic by such machines, 
both natural and artificial. Here the chief work is that of 
Turing?. We have said before that the machina ratliocinatriv 
is nothing but the calculus ratiocinalor of Leibniz with an en- 
gine in it ; and just as modern mathematical logic begins with 
this calculus, so it is inevitable that ils present engineering 
development should cast a new light on logic. The science of 
loday is operational : that is, it considers every stalement as es- 
sentially concerned with possible experiments or observable 
Processes. According to this, the study of logic must reduce to 
the study of the logical machine, whether nervous or mechani- 
cal, with all ils non-removable limitations and imperfections. 

It may be said by some readers that this reduces logic to psy- 
Chology, and thal the two sciences are observably and demon- 
Strably different. This is true in the sense that many psycholo- 
gical states and sequences of thought do not conform to the 
Canons of logic. Psychology contains much that is foreign to 
logic, but — and this is the important fact — any logic which 
means anything to us can contain nothing which the human 
mind — and hence the human nervous system — is unable to 
encompass. All logic is limited by the limitations of ihe human 
mind, when it is engaged in thal activily known as logical thinking. 

For example, we devote much of mathematics to discussions 
involving the infinite, but these discussions and their accom- 
Panying proofs are not infinite in fact. No admissible proof 
involves more than a finite number of stages. It is true, a proof 


by mathematical induction seems to involve an infinily of stages, 


but this is only apparent. In fact, it involves just the following 


Stages ; 
(1) Px is a proposition involving the number n; 
(2) P, has been proved forn = 1; 
(3) If P, is true, Pn4i is trues > 
(4) Therefore P, is true for every positive integer n. 


1. Loe. cit. 
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It is true that somewhere in our logical assumptions a. 
must be one which validates this argument. However, é 
mathematical induction is a far different thing sean es 
plete induction over an infinite set. The same thing 18 i 
of the more refined forms of mathematical induction, suc L . 
transfinite induction, which occur in certain mathematica 
disciplines. 

Thus some very interesting situations arise, in which we ie. 
be able -— with enough time and enough computational aids = a 
prove every single case of a theorem Pn; but if there is no Be 
tematic way of subsuming these proofs under a single argume ,. 
independent of n, such as we find in mathematical induction, a 
may be impossible to prove Pn jor all n. This contingency - 
recognized in what is known as metamathematics, the disciplin 
so brilliantly developed by Gédel and his school. A 

A proof represents a logical process which has come to £ 
definitive conclusion in a finite number of stages. Howevels a 
logical machine following definite rules need never come Us fi 
conclusion. It may goon grinding through different stages wie 
out ever coming to a stop, either by describing a pattern © 
activity of continually increasing complexity, or by going into @ 
repetitive process like the end of a chess game in which there }§ 
a continuing cycle of Perpetual check. This occurs in the case 


of some of the paradoxes of Cantor and Russell. Let us consider 
the class of all classes which 


Is this class a member of its 


» NO», « yes », « no », an 
equilibrium. 

Bertrand Russell’s solution of his own Paradoxes was to affix 
to every statement a quantity, the SO-called type, which serves 
to distinguish between what seems to be formally Mioleame stale: 
ment, according to the character of the objects with which it 
concerns itself ; — whether these are « things », in the simplest 
sense, Classes of « things », classes of Classes of « things », et¢- 
The method by which we resolve the Paradoxes jis also to 
attach a parameter to each Statement 


rete » this parameter being the 
time at whichit is asserted. In both Cases, we introduce what 
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we may call a parameter of uniformization, to resolye an ambi- 


guity which is simply due to its neglect. 

We thus see that the logic of the machine resembles human 
logic, and following Turing, we may employ it to throw light on 
human logic. Has the machine a more eminently human char- 
acteristic as well — the ability to learn 2 To see that it may 
well have even this property, let us consider two closely related 
notions : that of the association of ideas and that of the condi- 
tioned reflex. 

In the British empirical school of p 
Hume, the content of the mind was considered to be made up of 
certain entities known to Locke as ideas, and to the later authors 
as ideas and impressions. The simple ideas or impressions 
Were supposed to exist in a purely passive mind, as free from 
influence on the ideas it contained as a clean blackboard is on 
the symbols which may be written on it. By some sort ofinner 
activity, hardly worthy to be called a force, these ideas were sup- 
Posed to unite themselves into bundles, according to the prin- 
Ciples of similarity, contiguity, and cause and effect. Of these 
Principles perhaps the most significant was contiguity : ideas or 
impressions which had often occurred together in time or in space 
Were supposed to have acquired the ability of evoking one ano- 
ther, so that the presence of any one of them would produce 
the entire bundle. : 

Tn all this there is a dynamics implied, but the idea of a dyna- 
mics had not yet filtered through from physics to the biological 
and psychological sciences. The typical biologist of the eight- 
eenth century was Linnaeus, the collector and classifier, with a 
Point of view quite opposed to that of the evolutionists, the phy- 
Siologists, the geneticists, the experimental embryologists of the 
Present day. Indeed, with so much of the world to explore, the 
state of mind of the biologists could hardly have been different. 
Similarly, in psychology, the notion of mental content domin- 
ated that of mental process. This may well have been asurvival 
of the scholastic emphasis on substances, ina world in which the 
noun was hypostasised and the verb carried little or no weight. 
Nevertheless, the step from these static ideas to the more dyna- 
mic point of view of the present day, aS exemplified in the 
work of Pavlov; is perfectly clear. 


nilosophy, from Locke to 
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Pavlov worked much more with animals than with men, ong 
he reported visible actions rather than introspective poe. 
mind. He found in dogs that the presence of food causes t 4 
increased secretion of saliva and of gastric juice. Ifthena 7 
tain visual object is shown to dogs in the presence of food, ae 
only in the presence of food, the sight of this object in at 
absence of food will acquire the property of being by itself a b 
to stimulate the flow of saliva or of gastric juice. Theunion ie 
contiguity which Locke had observed introspectively m er 
case of ideas now becomes a similar union of patterns of beha 
vior. 

There is one important difference, how 
of view of Pavloy 
this fact that Loe 
action. Ther 


ever, between the point 
and that of Locke, and it is precisely due - 
ke considers ideas and Pavlov patterns a 
esponses observed by Pavlov tend to carry a pro 
cess to a successful conclusion, or to ayoid a catastrophe: 
Salivation is important for deglutition and for digestion, while 
the avoidance of what we should consider a painful stimulus 
lends to protect ‘he animal from bodily injury. Thus Be 
enters into the conditioned reflex something that we may C4 

ajfective tone. We need not Associate this with our own sensa 
tions of pleasure and pain, nor need we in the abstract associale 
it with the advantage of the animal. The essential thing is this: 


s 3 ga- 
that affective tone js arranged on some sort of scale from neg' 
tive « pain » to Posiliv 


+ arable 

© «pleasure » ; that for a considerablt 

lime, or permanently, an increase jin affective tone favors 
processes in the ner 


vous system that are under way at the time 


and gives them a Secondary power to increase affective tone: 
and that a decrease in affectiv. 


é € tone tends to inhibit all processes 

BAES way AU thes Cnnet waa gives them 4 secondary ability !® 
decrease affective tone. a 

Biologically speaking, of cours eater affective tone must 
occur predominantly in situations favorable be 
of the race, if not the individual] 
in situations which are unfavorable oe 
disastrous. Any race not conformin 
go the way of Lewis Carroll’s Bread- 
ate, Nevertheless, even a doomed ; 
valid so long as the race lasts, 


€, a gr 


the perpetuatio? 
aller affective tone 
S perpetuation, if not 
§ to this requirement will 
and-Butter Fly, and always 
ace may show a mechanis™ 
In other Words, even the most 


>and a sm 
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suicidal apportioning of affective tone will produce a definite 


pattern of conduct. 
Note that the mechanism of affective tone is itself a feed-back 


mechanism. It may even be givena diagram such as the fol- 
lowing. 


Process 3 


Affective tone 
mechanism 


Affective tone 
mechanism 


Affective 
tone totalizer 


Process 1 


Affective tone 
mechanism 


Fic. 7. 


for affective tone combines the affective 
tones given by the separale affective-tone mechanisms over 
a short interval in the past, according to some rule which we 
need not specify now. The leads back to the individual affec- 
tive tone mechanisms serve to modify theintrinsic affective tone 
of each process in the direction of the output of the lotalizer, 
and this modification stands until it is modified by later messages 
from the totalizer. ‘The leads back from the totalizer to the 
process mechanisms serve to lower thresholds if the total affec- 
tive tone is increasing, and to raise them if the total affective 
tone is decreasing. They likewise have a long-time effect, which 
endures until it is modified by another impulse from the totali- 
zer. his lasting effect, however, is confined to those processes 
actually in being at the time the return message arrives, and a 
Similar limitation also applies to the effects on the individual 
affective-tone mechanisms. 

I wish to emphasize that Ido not say that the process of the 
conditioned reflex operates according to the mechanism I have 
given ; I merely say that it could so operate. If, however, we 
assume this or any simular mechanism, there are a good many 


Here the totalizer 
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things we can say concerning it. One is that this mechanism is 
capable of learning. It has already been recognized that the 
conditioned reflex is a learning mechanism, and this idea has 
been used in the behaviorist studies of the learning of rats in a 
maze. All thatis needed is that the inducements or punishments 
used have respectively a positive and a negative affective tone. 
This is certainly the case, and the experimenter learns the UES 
ture of this affective tone by experience, not simply by a priori 
considerations. 

Another point of considerable interest is that such a mecha- 
nism involves a certain set of messages which go out gene- 
rally into the nervous system, to all elements which are in a 
Stale lo receive them. These are the return messages from the 
affective-tone totalizer, and to a certain extent, the messages 
from the affective-tone mechanisms to the totalizers. Indeed, 
the tolalizer need not be a separate element, but may merely 
represent some natural combinatory effect of messages arriving 
from the individual affective-tone mechanisms. Now, such mes- 
Sages « to whom it may concern » may well be sent out most 
efficiently, witha smallest cost in apparatus, by channels other 
than nervous. In a similar manner, the ordinary communica- 
tion system of a mine may consist ofa telephone central with 
the attached wiring and pieces of apparatus. When we want to 
empty a mine in a hurry, we do not trust to this, but break a 


tube of a mercaptan in the air intake. Chemical messengers 
like this, or like the hor 


tive for a message not 


monal activity is most Suggestive. This does not mean that a 
purely nervous mechanism is not capable of affective tone and 
of learning, but it does mean that in the study of this aspect of 
our mental activity, we cannot afford to be blind to the possi- 
bilities of hormonal transmission. [t may be excessively fanci- 
ful to attach this notion to the fact that in the theories of Freud, 
the memory — the Storage function of the nervous system — 
and the activities of sex are bothinyolved, Sex, on the one hand, 
and all affective content, on the other, contain a very strong 
hornional element. This suggestion of the importance of sex 
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and hormones has been made to me by Dr. J. Lettvin and 
Mr. Oliver Selfridge. While at present there is no adequate 
evidence to prove its validity, it is not manifestly absurd in prin- 
ciple. 

There is nothing in the nature of the computing machine which 
forbids it to show conditioned reflexes. Let us remember that 
a computing machine in action is more than the concatenation 
of relays and storage mechanisms which the designer has built 
into it. It also contains the content of its storage mechanisms; 
and this content is never completely cleared in the course of a 
single run. We have already seen that it is the run rather than 
the entire existence of the mechanical structure of the computing 
machine which corresponds to the life of the individual. We 
have also seen that in the nervous computing machine, itis 
highly probable that information is stored largely as changes in 
the permeability of the synapses ; and it is perfectly possible 
to construct artificial machines where information is stored 
in that way. It is perfectly possible, for example, to cause any 
message going into storage to change in a permanent or semi- 
permanent way the grid bias of one or of a number of vacuum 
tubes, and thus to aller the numerical value of the summation 
of impulses which will make the tube or tubes fire. 

A more detailed account of learning apparatus in computing 
and control machines, and the uses to which it may be put, may 
well be left to the engineer rather than toa preliminary book 
like this one. It is perhaps better to devote the rest of this 
chapter to the more developed, normal uses of modern computing 
machines. One of the chief of these is in the solution of partial 
Even linear partial differential equations 
require the recording of an enormous mass of data to set them 
up, as the data involve the accurate description of functions of 
two or more variables. With equations of the hyperbolic type, 
like the wave equation, the typical problem is that of solving the 
equation when the initial data are given, and this can be done 
in a progressive manner from the initial data to the results at 
any given later time. This is largely true of equations of the 
parabolic type as well. Whenit comes toequations ofthe elliptic 
type, where the natural data are boundary values rather than 
initial values, the natural methods of solution involve an itera- 


differential equations. 
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€ process of successive approximation. This process is 
peated a very large number oftimes, so that very fast methods 
ch as those of the modern computing machine are almost indis- 
nsable. 

In non-linear partial differential equations, we miss what 
- have in the case of the linear equations —a reasonably ade- 
ate purely mathematical theory. Here computational me- 
ods are not only important for the handling of particular 
merical cases, but, as von Neumann has pointed out, we need 
2m in order to form that acquaintance with a large number 
particular cases without which we can scarcely formulate a 
neral theory. To some extent this has been done with the aid 
very expensive experimental apparatus, such as wind tun- 
ls. It is in this way that we have become acquainted with 
> more complicated properties of shock waves, slip surfaces, 
rbulence, and the like, for which we are scarcely in 
give an adequate mathematical theory. How m 
vered phenomena of similar nature there may be, we do not 
low. The analogy machines are so much less accurate, and 
many cases so much slower, than the digital machines, that 
e latter give us much more promise for the future. 

It is already becoming clear i 
at they demand purely m 
lite different from those 


a position 
any undis- 


nthe use of these new machines 
athematical techniques of their own, 

in use in manual computation or in 
€ use of machines of smaller Capacity. For example, even the 
e of machines for computing determinants of moderately high 
der, or for the simultaneous solution of twenty or thirty si- 
ultaneous linear equations, shows difficulties which do notarise 
1en we study analogous problems of small order. 
reis exercised in setting a problem u 
prive the solution of any significan 
commonplace to gs 


Unless 


place in the hands of those 
it possessing a sufficient degree o 


f technical skill to take full 
vantage of them. The ultra-r 


apid computing machine will 
rtainly not decrease the need f 


or mathematicians with a high 
el of understanding and technical training. 


n the mechanical or electrical construe 


lion of computing 
chines, there are afew maxims which des 


erve consideration. 
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One is that mechanisms which are relatively frequently used 
such as multiplying or adding mechanisms, should be in the 
form of relatively standardized assemblages adapted for ont 
particular use, and no other ; while those of more occasiona 
use should be assembled for the moment of use out of element: 
also available for other purposes. Closely related to this con 
sideration is the one that in these more general mechanisms, thi 
component parts should be available in accordance with thei 
general properties, and should not be alloted permanently t 
a specific association with other pieces of apparatus. Ther 
should be some part of the apparatus, like an automatic telephone 
switching exchange, which will search for free components an 
connectors ofthe varioussorts, andallotthem as they are needed 
This will eliminate much of the very large expense which is du 
to having a great number of unused elements, which cannol b 
used unless their entire large assembly is used. We shall fin 
this principle is very important when we come to conside 
traffic problems and overloading in the nervous system. 

Asa final remark, let me point out that a large computin 
machine, whether in the form of mechanical or electric apparatu 
orin the form of the brain itself, uses up & considerable amoun 
of power, all of which is wasted and dissipated in heat. Th 
blood leaving the brain is a fraction of a degree warmer thai 
that entering it. No other computing machine approaches th 
economy of energy of the brain. In a large apparatus like th 
Eniac or Edvac, the filaments of the tubes consume a quantit 
of energy which may well be measured in kilowatts. and unle: 
adequate ventilating and cooling apparatus is provided, th 
system will suffer from what is the mechanical equivalent « 
pyrexia, until the constants of the machine are radicall 
changed by the heat, and its performance breaks down. Neve 
theless, the energy spent per individual operation is almost v: 
nishingly small, and does not even begin to form an adequa 
measure of the performance of the apparatus. The mechanic: 
brain does not secrete thought « as the liver does bile », as th 
earlier materialists claimed, nor’ does it put it out in the for 
of energy, as the muscle puls out its activity. Information | 
information, not matter or energy- No materialism which dos 


not admit this can survive at the present day. 


CHAPTER VI 
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Among other things which we have discussed in the last chapter 
is the possibility of assigning a neural mechanism to Locke's 
theory of the association of ideas. According to Locke, this 
occurs according to three principles: the principle of contiguity, 
the principle of similarity, and the principle of cause and effect. 
The third of these is reduced by Locke, and even more defini- 
tively by Hume, to nothing more than constant concomitance, 
and so is subsumed under the first, that of contiguity. The 
second, that of similarity, deserves a more detailed discussion. 

How do we recognize the identity of the features of a man, 
whether we see him in profile, in three-quarters face, or in full 
face? How do we recognize acircleasa circle, whether it is large 
or small, near or far ; whether in fact itis in a plane perpendicular 
to a line from the eye meeling it in the middle, and is seen as a 
circle, or has some other orientation, and is seen as an ellipse ? 
How do we see faces and animals and maps in clouds, or in the 
blots of a Rorschach test ? All these examples refer to the eye, 
but similar problems extend to the other senses 
them have to do with inter-sensory relations 
into words the call of a bird or the stridulation 
How do we identify the roundnes: 

For the present let us confine ourselves to the sense of 
vision. One important factor in the comparison of form of diffe- 
rent objects is certainly the interaction of the eye and the mus- 
cles, whether they are the muscles within the eyeball, the mus- 
cles moving the eyeball, the muscles Moving the head, or the 
muscles moving the body as a whole. Indeed, some form of 
this visual-muscular feed-back system is important as low in the 
animal kingdom as the flatworms. There the negative photo- 


, and some of 
How do we put 
s of an insect ? 
$ of a coin by touch ? 
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tropism, the tendency to avoid the light, seems to be controlled 
by the balance of the impulses from the two eyespots. This 
balance is fed back to the muscles of the trunk, turning the 
body away from the light, and in combination with the general 
impulse to move forward, carries the animal into the darkest 
region accessible. It is interesting to note that a combination 
of a pair of photocells with appropriate amplifiers, a Wheatstone 
bridge for balancing their outputs, and further amplifiers con- 
trolling the input into the two motors of a twinscrew mecha- 
nism, would give us a very adequate negatively phototropic 
control for a little boat. It would be difficult or impossible 
for us to compress this mechanism into the dimensions that 
a flatworm can carry ;but here we merely have another exem- 
plification of the fact that must by now be familiar to the reader, 
that living mechanisms tend to have a much smaller space scale 
than the mechanisms best suited to the techniques of human 
artificers, although on the other hand the use of electrical tech- 
niques gives the artificial mechanism an enormous advantage 
in speed over the living organism. 

Without going through all the intermediate stages, let us come 
at once to the eye-muscle feed-backs in man. Some of these are 
of purely homeostatic nature, aswhen the pupil opens in the dark 
and closes in the light, thus tending to confine the flow of light 
into the eye between narrower bounds than would olherwise 
be possible. Others concern the fact that the human eye has 
economically confined its best form and color vision to a rela- 
tively small fovea, while its perceplion of motion is better on the 
periphery. When the peripheral vision has picked up some 
object conspicuous by brilliancy or light-contrast or color or 
above all by motion, there is a reflex feed-back to bring it into 
the fovea. This feed-back is accompanied by a complicated 
system of interlinked subordinate feed-backs, which tend to con- 
verge the two eyes so thatthe object attracting attention is in the 
same part of the visual field of each, and to focus the lens so thatits 
outlines are as sharp as possible. These actions are supplemen- 
ted by motions of the head and body, by which we bring the 
object into the center of vision if this cannot be done readily 
by a motion of the eyes alone, or by which we bring an object 
outside the visual field picked up by some other sense into that 
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field. In the case of objects with which we are more familiar in 
One angular orientation than another — writing, human faces, 
landscapes, and the like — there is also a mechanism by which 
we tend to pull them into the proper orientation. 

All these processes can be summed up in one sentence: we tend 
to bring any object that attracts our attention into a standard 
position and orientation, so that the visual image which we form 
of it varies within as small a range as possible. This does not 
exhaust the processes which are involved in perceiving the form 
and meaning of the object, but it certainly facilitates all later 
processes tending to this end. These later processes occur in 
the eye and in the visual cortex. There is considerable evidence 
that for a considerable number of stages, each step in this pro- 
cess diminishes the number of neuron channels involved in the 
transmission of visual information, and brings this information 
one step nearer to the form in which it is used and is preserved 
in the memory. 

The first step in this concentration of visual information occurs 
in the transition between the retina and the opticnerve. It willbe 
noted that while in the fovea there is almost a one-one correspon- 
dence between the rods and cones and the fibres of the optic nerve, 
the correspondence on the periphery is such th 
fibre corresponds to ten or mor 
understandable, in view of the fact that the chief function of the 
peripheral fibres is not so much vision itself as a pick-up for 
the cenlering and focussing-directing mechanism of the eye. 

; One of the most remarkable phenomena of vision is our abi- 
lity to recognize an outline drawing. Clearly an outline drawing 
of, ENO the face of a man, has very little resemblance to the face 
itself in color, or in the massing of light and shade 3 yet it may 
be a most recognizable portrait of its subject. The most plau- 
sible explanation of this is that, Somewhere in the visual process, 
SBiaies in emphasized and some other aspects of an image 
ave Mupunied in importance. The beginning of these processes 
isin the eye itself. Like all Senses, the retina is subject to acco- 
modation : thatis, the constant maintenance of a stimulus reduces 
its ability to receive and to transmit that stimulus. 
markedly so for the receptors which record the interi 
block of images with constant color and illumination 


atone optic nerve 
e end organs. This is quite 


This is most 
or of a large 
, for even the 
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slight fluctuations of focus and point of fixation which are inevi- 
table in vision do not change the character of the image received. 
Itis quite different on the boundary of two contrasting regions. 
Here these fluctuations produce an alternation between one sti- 
mulus and another, and this alternation, as we see in the pheno- 
menon of after-images, not only does not tend to exhaust the 
visual mechanism by accomodation, but even tends to enhance 
its sensitivity. This is true whether the contrast between the 
two adjacent regions is one of light-intensity or of color. Asa 
comment on these facts, let us note that 3/4 of the fibers in the 
optic nerve respond only to the flashing « on » of illumination. 
We thus find that the eye receives its most intense impression 
at boundaries, and that every visual image in fact has something 
of the nature of a line drawing. 

Probably not all of this action is peripheral. In photogra- 
phy, it is known that certain treatments of a plate increase its 
contrasts, and such phenomena, which are of non-linearity, are 
certainly not beyond what the nervous system can do. They 
are allied to the phenomena of the telegraph-type repeater, 
which we have already mentioned. Like this, they use an im- 
pression which has nol been blurred beyond a certain point to 
trigger a new impression of a standard sharpness. At any 
rate, they decrease the total unusable information carried by an 
image, and are probably correlated with a part of the reduction — 
of the number of transmission fibres found at various stages 
of the visual cortex. 

We have thus designated several actual or possible stages of 
the diagrammatization of our visual impressions. We center 
our images around the focus of attention, and reduce them more 
or less to outlines. We have now to compare them with one 
another, or at any rate with a standard impression stored 


in memory, such as « circle » or «square ». This may be done 
We have given a rough sketch which indicates 
f contiguity in association may be 
he principle of contiguity also 


in several ways. 
how the Lockean principle © 
mechanized. Let us notice that t 
covers much of the other Lockean principle of similarity. The 
different aspects of the same object are often to be seen in those 
processes which bring it to the focus of attention, and of other 
motions which lead us to see it, NOW at one distance and now at 
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another, now from one angle and now from a distinct one. This 
is a general principle, not confined inits application to any pattie 
cular sense, and doubtless of much importance in the comparison 
of our more complicated experiences. It is nevertheless proba- 
bly not the only process which leads to the formation of our 
specifically visual general ideas, or as Locke would call them, 
« complex ideas ». The structure of our visual cortex is too 
highly organized, too specific, to lead us to suppose that it operates 
by what is after all a highly generalized mechanism. It leaves 
us the impression that we are here dealing with a special mecha- 
nism which is not merely a temporary assemblage of general- 
purpose elements, with interchangeable parts, but a permanent 
sub-assembly like the adding and multiplying assemblies of a 
computing machine. Under the circumstances, it is worth con- 
sidering how such a sub-assembly might possibly work, and how 
we should go about designing it. 

The possible perspective transformations of an object form 
what is known asa group in the sense in which we have already 
defined one in Chapter II. This group defines several sub- 
groups of transformations : the affine group, in which we consi- 
der only those transformations which leave the region at infi- 
nity untouched ; the homogeneous dilations about a given point, 


in which one point, the directions of the axes, and the equality 


of scale in all directions are preserved; the transformations pre- 
serving length ; 


; the rotations in two or three dimensions about 
a point ; the set of all translations, and so on. Among these 
groups the ones we have just mentioned are continuous: that 
is, the operations belonging to them are determined by the va- 
lues of a number of continuously varying parameters in an ap- 
propriate space. ‘They thus form multi-dimensional configura- 
tions in n-space, and contain sub-sets of transformations which 
constitute regions in sucha space. 

Now, just as a region in the ordinary two-dimensional plane 
is covered by the process of scanning known to the televi- 
sion engineer, by which a nearly uniformly distributed set of 
sample positions in that region is taken to represent the whole, 
so every region in a group-space, including the whole of such a 
space, can be represented by a process of group scanning. In 
such a process, which is by no means confined to a space of 
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three dimensions, a net of positions in the space is traversed 
in a one-dimensional sequence, and this net of positions is so 
distributed that it comes near to every position in the region, 
in some appropriately defined sense. It will thus contain posi- 
tions as near to any we wish as may be desired. If these « po- 
sitions », or sets of parameters, are actually used to generale 
the appropriate transformations, it means that the results of 
transforming a given figure by these transformations will come 
as near as we wish to any given transformation of the figure by 
a transformation operator lying in the region desired. If our 
scanning is fine enough, and the region transformed has the 
maximum dimensionality of the regions transformed by the group 
considered, this means that the transformations actually tra- 
versed will give a resulting region overlapping any transform 
of the original region by an amount which is as large a fraction 
of its area as we wish. 

Let us then start with a fixed comparison region and a region 
to be compared with it. If at any stage of the scanning of the 
group of transformations the image of the region to be compared 
under some one of the transformations scanned coincides 
more perfectly with the fixed pattern than a given tolerance 
allows, this is recorded, and the two regions are said to be alike. 
If this happens at no stage of the scanning process, they are said 
to be unlike. This process is perfectly adapted to mechaniza- 
tion, and serves as a method to identify the shape of a figure 
independently of its size or its orientation or of whatever 
transformations may be included in the group-region to be 


scanned. 

If this region is not the enti 
gion A seems like region B, an 
C, while region A does not seem 
happens in reality. A figure ™ 


semblance to the same figure inv : 2 
immediate impression — one not involving any of the higher 


processes — is concerned. Nevertheless, at each stage of its in- 
version, there may be a considerable range of neighboring posi- 
tions which appear similar. The universal « ideas » thus 


formed are not perfectly distinct, but shade into one another. 
There are other more sophisticated means of using group 


re group, it may well be that re- 
d that region B seems like region 
like region C. This certainly 
ay not show any particular re- 
erted, at least in so far as the 
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scanning to abstract from the transformations of a group. 
The groups which we here consider have a « group measure », 
a probability density which depends on the transformation 
group itself, and does not change when all the transformations 
of the group are altered by being preceded or followed by any 
specific transformation of the group. Itis possible to scan ihe 
group in such a way that the density of scanning of any region 
of a considerable class — that is the amount of time which the 
variable scanning element passes within the regionin any com- 
plete scanning of the group— is closely proportional to its group 
measure. In the case of such a uniform scanning, if we have 
any quantity depending on a set S of elements transformed by 
the group, and if this set of elements is transformed by all the 
transformations of the group, let us designate the quantity de- 
pending on S by Q} S}, and let us use TS to express the 
transform of the set S by the transformation T of the group. 
Then Q | TS | will be the value of the quantity replacing Q| S| 
when S is replaced by TS. If we average or integrate this with 
respect to the group measure for the group of transformations T, 


we shall obtain a quantity which we may wrile in some such 
form as. 


(6.01) [2 | TS | at, 


nse the same form or Gestalt. It is 
possible to obtain an approximate comparability of form where 
the integration in (6.01) is over less than the whole group, if 


the integrand Q } TS fis small over the region omitted. So 
much for group measure. 


In recent years, 


blind, to work by the use 


We shall Suppose these efforts con- 
fined to printed matter, and even to a single type-face, or to a 


small number of type-faces. We shall also suppose that the 
alignment of the page, the centering of the lines, the traverse 
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from line to line taken care of either manually, or as they may 
well be, automatically. These processes correspond, as we may 
see, to the part of our visual Gestalt determination which depends 
on muscular teed-backs and the use of our normal centering, 
orienting, focussing, and converging apparatus. There now en- 
sues the problem of determining the shapes of the individual 
letters as the scanning apparatus passes over them in sequence. 
It has been suggested that this be done by the use of several pho- 
to-electric cells placed in a vertical sequence, each attached to a 
sound-making apparatus of a different pitch. This can be done 
with the black of the letters registering either as silence or as 
sound. Let us assume the latter case, and let us assume three 
photocell receptors above one another. Let them record as 
the three notes of a chord, let us say, with the highest note on 
top and the lowest note below. Then the letler capital F, let 
us say, will record : 

Duration of upper note ; 
Duration of middle note ; 
Duration of lower nole. 


The letter capital Z will record : 


the letter capital O: 


iven by our ability lo 
ad such an audilory 
instance. 


and soon. With the ordinary help g 
interpret, it should not be too difficult to re 
code : not more difficult than to read Braille, for 

However, all this depends on one thing : the proper re 
of the photocells to the vertical height of the letters. Even 
with standardized type faces, there still are great variations 
in the size of the type. ‘Uhus it is desirable for us to be able to 
pull the vertical scale of the scanning up or down, in order to 
reduce the impression of a given letter toa standard. We must 


lation 
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at least have at our disposal, manually or automatically, some 
of the transformations of the vertical dilation group. 

‘There are several ways we might do this. We might allow 
for a mechanical vertical adjustment of our photocells. ee! 
the other hand, we might use a rather large vertical anray, a 
photocells, and change the pitch-assignment with the S126 ° 
type, leaving those above and below the type silent. This — 
be done, for example, with the aid of a schema of two sets 0 
connectors, the inputs coming up from the photocells, and lea- 
ding to a series of switches of wider and wider Givengencs 
and the outputs a series of vertical lines, as in the following: 


Layer of oscillators 


Layer of eel 


Here the single lines re 


Fic. 8. 


present the leads from the photocells, 
the double lines the leads lo the oscillators, the circles on the 


dotted lines the points of connections between incoming and 
outgoing leads, and the dotted lines themselves the leads where- 
by one or another of a bank of Oscillators is put into action. 
This was the device, to which we have referred in the introduc- 
tion, designed by McCulloch for the purpose of adjusting to the 
height of the type-face. In the first design the selection be- 
tween dotted line and dotted line was manual. 

This was the figure which, when shown to Dr. von Bonin, 


suggested the fourth layer of the visual cortex. It was the 
connecting circles which s 


this layer, arranged in sub-layers of 


uniformly changing hori- 
zontal density, 


the Opposite direction to 
the density. The horizontal leads are probably fired in some 


cyclical order. The whole apparatus seems quile suited to the 


process of group scanning. There must of course be some pro- 
cess of recombination in time of the upper outputs. 


This then was the device Suggested by McCulloch as that 


and size changing in 


Ee 
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actually used in the brain in the detection of visual Gestalt. 
It represents a type of device usable for any sort of group scan- 
ning. Something similar occurs in other senses as well. In 
the ear, the transposition of music from one fundamental pitch 
to another is nothing but a translation of the logarithm of the 
frequency, and may consequently be performed by a group- 
scanning apparatus. 

A group-scanning assembly thus has well-defined appropriate 
anatomical structure. Thenecessary switching may be performed 
by independent horizontal leads which furnish enough stimu- 
lation to shift the thresholds in each level to just the proper 
amount to make them fire when the lead comes on. While we 
do not know all the details of the performance of the machinery, 
it is not at all difficult to conjecture a possible machinery con- 
forming to the anatomy. In short, the group-scanning assembly 
is well adapted to form the sort of permanent sub-assembly of 
the brain corresponding to the adders or multipliers of the 


numerical computing machine. 
Lastly, the scanning apparatus s 
period of operation which should be i 
mance of the brain. The order of magnitude of this 
should show in the minimum time required for making direct 
comparison of the shapes of objects different in size. This can 
only be done when the comparison is between two objects not 
too different in size; otherwise, it is a long-time process, sugges- 
tive of the action of a non-specific assembly. When direct com- 
parison seems to be possible, it appears to take a time of the 
order of magnitude of a tenth of asecond. This also seems to 
accord with the order of magnitude of Ihe time needed by exci- 
tation to stimulate all the layers of transverse connectors in 


cyclical sequence. 

While this cyclical process then might be a locally determined 
one, there is evidence that there is a wide-spread synchronism 
in different parts of the cortex, suggesting that it is driven from 
some clocking center. In fact, it has the order of frequency 
appropriate for the alpha rhythm of the brain, as shown in elec- 
troencephalograms. We may suspect that this alpha rhythm is 
associated with form perception, and that it partakes of the 
nature of a sweep rhythm, like the rhythm shown in the scanning 


hould have a certain intrinsic 


dentifiable in the perfor- 
period 
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process of a television apparatus. It disappears in deep sleep, 
and seems to be obscured and overlaid with other rhythms, 
precisely as we might expecl, when we are actually looking at 
something, and the sweep rhythm is acting as something like a 
carrier for other rhythms and activities. It is most marked 
when the eyes are closed in waking, or when we are staring 
inlo space at nothing in particular, as in the condition of abstrac- 
tion of a Yogi, when it shows an almost perfect periodicity. 
We have just seen that the problem of sensory prosthesis — 
the problem of replacing the information which is normally 
conveyed by a lost sense by one which is still available — is 


important, and not necessarily insoluble. What makes it more 
Nopeful is the fact that the me 


mally approached throu 
key, but are avail 


only retains visual memor 
bul is even able to store t 
visual form. He may feel his way 
an image of how it ought to look. 

Thus a part of his nor 
him. On the other han 


auditory impressions in a 
around a room, and yet have 


a replacement of 
sight by hearing is at least in part a Comparison between the 
number of recognizably different visual patterns and tecogniza- 
bly different auditory patterns aj the cortical level. This is a 
comparison of amounts of information, In view of the some- 
what similar organization of the different parts of the sensory 
cortex, it will probably not differ very much from a comparison 


1. Personal communication of Dr. Walter, of Bristol, England. 
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between the areas of the two parts of the cortex. This is 
about 100: 1 as between sight and sound. If all the auditory 
cortex were used for vision, we might expect to get a quantity 
of reception of information about one per cent of that coming 
in through the eye. On the other hand, our usual scale for the 
estimation of vision is in terms of the relative distance al which 
a certain degree of resolution of pattern is obtained, and thus 
a 10/100 vision means an amount of flow of information about 
One per cent of normal. This is very poor vision : it is, howe- 
ver, definitely not blindness, nor do people with this amount of 
vision necessarily consider themselves as blind. 

In the other direction the picture is even more favorable. 
The eye can detect all of the nuances of the ear with the use of 
Only one per cent of its facilities, and still leave a vision of 
about 95/100, which is substantially perfect. Thus the problem 
of sensory prosthesis is an extremely hopeful field of work. 


CHAPTER VIL 
CYBERNETICS AND PSYCHOPATHOLOGY 


It is necessary that [ commence this chapter with a disavowal. 
On the one hand, Iam not a psychopathologist nor a psychia- 
trist, and lack any experience in a field where the guidance of 
experience is the only trustworthy one. On the other hand, our 
knowledge of the normal performance of the brain and the 
nervous system, and a fortiori,our knowledge of their abnormal 
performance, is far from having reached that state of perfection 
where an a priori theory can command any confidence. I 
therefore wish to disclaim in advance any assertion that any 
particular entity in psychopathology, as for example any of the 
morbid conditions described by Krae 
due to a specific type of defect in the 
as a computing machine, 
conclusions from the consi 
own risk. 

Nevertheless, the realiz 


pelin and his disciples, is 
organization of the brain 
Those who may draw such specific 
derations of this book do so at their 


ation that the brain and the computing 
machine have much in common may Suggest new and _ valid 
approaches to psychopathology, and even to psychiatrics. 
These begin with perhaps the simplest question of all : how the 
brain avoids gross blunders, gross miscarriages of activity, due 
to the malfunction of individual components. Similar questions 
referring to the computing machine are of great practical impor- 
tance, for here a chain of Operations each covering a fraction of 
a millisecond may last a matter of hours or days. It is quite 
possible for a chain of computational 
10° separate steps. Under these circumst 
least one operation will go amiss is ver 
though, it is true, the reliability of mo 
has far exceeded the most sanguine e 


Operations to involve 
ances, the chance thatat 
y far from negligible,even 
dern electronic apparatus 
xpectations. 
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In ordinary computational practice by hand or by desk 
machines, it is the custom to check every step of the computa- 
s found, to localize it by a backward 


tion, and when an error i 
here the error is noted. 


process starting from the first point w: 
To do this with a high-speed machine, the check must proceed 
with the speed of the original machine, or the whole effective 
order of speed of the machine will conform to that of the slower 
process of checking. Furthermore, if the machine is made to 
keep all intermediate records of its performance, its complica- 
tion and bulk will be increased to an intolerable point, by a 
factor which is likely to be enormously greater than two or three. 

A much better method of checking, and in fact the one gene- 
rally used in practice, is lo refer every operation simultaneously 
to two or three separate mechanisms. In the case of the use of 
two such mechanisms, their answers are automatically collated 
against each other ; and if there isa discrepancy, all data are 
transferred to permanent storage, the machine stops, and a 
signal is sent to the operator that something is wrong: The 
Operator then compares the results, and is guided by them in 
his search for the malfunctioning part, perhaps a tube which 
has burnt out and needs replacement. If three separate mecha- 
nisms are used for each stage, and single misfunctions are as 
rare as they are in fact, there will practically always be an 
agreement between two of the three mechanisms, and this 
agreement will give the required result. In this case the colla- 
tion mechanism accepts the majorily report, and the machine 
need not stop; but there is a signal indicating where and how 
the minority report differs from the majorily report. If this 
occurs at the first moment of discrepancy, the indication of the 
position of the error may be very precise. Ina well-designed 
machine, no particular element is assigned to a particular stage 
in the sequence of operations, but at each stage there is a 
Searching process, quite similar to that used in automatic 
telephone exchanges, which finds the first available element of 
a given sort, and switches it into the sequence of operations. 
In this case, the removal and replacement of defective elements 
need not be the source of any appreciable delay. 

It is conceivable and not implausible that at least two of the 
elements of this process are also represented in the nervous 
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system. We can hardly expect that any important message 
is entrusted for transmission to a single neuron, nor that any 
important operation is entrusted to a single neuronal mechanism. 
Like the computing machine, the brain probably works on a 
variant of the famous principle expounded by Lewis Carroll in 
The Hunting of the Snark: « What I tell you three times is 
true». Itis also improbable that the various channels available 
for the transfer of information generally go from one end of 
their course to the other without anastomosing. It is much 
more probable that when a message comes in to a certain level 
of the nervous system, it may leav 
the next by one or more alternative members of what is known 
as an cinternuncial pool ». There may be parts of the nervous 
system, indeed, where this interchangeability is much limited or 
abolished, and these are likely to be such highly specialized 
parts of the cortex as those which serve as the inward exlen- 
sions of the organs of special sense. Still, the principle holds, 
and probably holds most clearly for the relatively unspecialized 
cortical areas which serve the purpose of association and of 
what we call the higher mental functions. 

So far we have been considering errors in performance which 
are normal,and only pathological in an extended sense. Let us 
now turn to those which are much more cl 
Psychopathology has been 
instinctive materialism of the 


e that point and proceed to 


early pathological. 
rather a disappointment to the 
doctors, who have taken the point 
nust be accompanied by material 
involved. It is true that specific 
tumors, clots, and the like, may 
mptoms, and that certain mental 


ain of a schizophrenic 
ypes, nor ofa manic-depressive 


‘eae These disorders we call functional ; 
and this distinction seems to contravene the dogma of modern 


materialism, that every disorder in function has some physiolo- 
gical or anatomical basis in the tissues concerned. 

This distinction between functional and organic disorders 
receives a great deal of light from the consideration of the com- 


CYBERNETICS AND PSYCHOPATHOLOGY 474 


puling machine. As we have already seen, it is not the empty 
physical structure of the computing machine that corresponds 
to the brain — to the adult brain, at least — but the combination 
of this structure with the instructions given it at the beginning 
of a chain of operations and with all the additional information 
stored and gained from outside in the course of this chain. 


This information is stored in some physical form—in the form of 
the form of circulating memories, 


with a physical basis which vanishes when the machine is shut 
t in the form of long-time 


down, or the brain dies, and par 

memories, which are stored in a way at which we can only guess, 
but probably also in aform with a physical basis which vanishes 
at death. There is no way yet known for us to recognize in 
the cadaver what the threshold of a given synapse has been in 
life ; and even if we knew this, there is no way we can trace out 
the chain of neurons and synapses communicating with this, and 
the significance of this chain as to the ideational content which it 
records. 

There is therefore nothing surprising in considering the func- 
tional mental disorders as fundamentally diseases of memory, 
of the circulating information kept by the brain in the active 
state, and of the long-time permeability of synapses. Even the 
grosser disorders such as paresis may producea large part of their 
effects, not so much by the destruction of tissue which they 
involve, and the alteration of synaptic thresholds, as by the 
secondary disturbances of traffic, the overload of what remains 
of the nervous system, and the re-routing of messages, which 
must follow such primary injuries. . 

In a system containing a large number of neurons, circular 
processes can hardly be stable for long periods of time. Either, 
as in the case of memories belonging to the specious present, 
they run their course, dissipate themselves, and die out, or they 
comprehend more and more neurons in their system, until they 
occupy an inordinate part of the neuron pool. This is what we 
should expect to be the case in the malignant worry which ac- 
companies anxiety neuroses. In such a case, il is possible that 
the patient simply does not have the room, the sufficient number 
ofneurons, to carry oul his normal processes of thought. Under 
such conditions, there may be less going on in the brain to load 


memory — but part of it is in 
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up the neurons not yet affected, so that they are all the more 
readily involved in the expanding process. Furthermore, the 
permanent memory becomes more and more deeply involved, 
and the pathological process which occurred at first at the level 
of the circulating memories may repeat ilself in a more intrac- 
table form atthe level of the permanent memories. Thus what 
startedas a relatively trivial and accidental reversal of stability, 
may build itself up into a process totally destructive to the or- 
dinary mental life. 

Pathological processes of a somewhat similar nature are not 
unknown in the case of mechanical or electrical computing 
machines. A tooth of a wheel may slip under just such con- 
ditions that no tooth with which it engages can pull it back 
into its normal relations, or a high-speed electric 


al computing 
machine may 


go into a circular process which there seems to 
be no way to stop. These contingencies may depend on a 
highly improbable instantaneous configuration of the system, 
and when remedied, may never — or 
themselves. However, when they occur, 
the machine out of action. 


very rarely — repeat 
they temporarily put 


How do we deal with these accidents in the use of the machine? 
The first thing which we try is to clear the machine of all infor- 
mation, in the hope that when it starts again with different data, 
the difficulty may not recur. Failing this, if the difficulty is in 
some point permanently or temporarily inaccessible to the 
clearing mechanism, we shake the machine, or if it is electrical, 
subject it to an abnormally large electrical impulse, in the 
hope that we may reach the inaccessible part, and throw it into 
a position where the false cycle of its activities will be inter- 
rupted. If even this fails, we may disconnect an erring part 
of the apparatus, for it is possible that what yet remains may 
be adequate tor our purpose. 

Now, there is no normal process except death which com- 
pletely clears the brain from all past impressions ; and after 
death, it is impossible to set it going again. Of all normal pro- 
cesses, sleep comes the nearest to 4 non-pathological clearing. 
How often we find that the best way to handle = complicated 
worry, or an intellectual muddle, is to sleep oy 


erit! However, 
sleep does not clear away the deeper memorie 


s, nor indeed isa 
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sufficiently malignant state of worry compatible with an adequate 
sleep. Weare thus often forced to resort to more violenttypes of 
intervenlion in the memory cycle. The more violent of these 
involve a surgical intervention into the brain, leaving behind it 
permanent damage, mutilation, and the abridgementof the powers 
cf the victim;as the mammalian central nervous system seems to 
possess no powers whatever of regeneration. The principal type 


ofsurgical intervention which has been practiced is known as pre- 


frontal lobotomy, and consists in the removal or isolation of a 
It hasrecently been 


portion of the prefrontal lobe of the cortex. 
having a certain vogue, probably not unconnected with the fact 
that it makes the custodial care of many patients easier. Let me 
remark in passing that killing them makes their custodial care 
still easier. However, prefrontal lobotomy does seem to have a 
genuine effect on malignant worry, not by bringing the patient 
nearer to a solution of his problems, but by damaging or des- 
troying the capacily for maintained worry, known in the termi- 
nology of another profession as the conscience. More generally 
it appears to limit all aspects of the circulating memory, the 
ability to keep in mind a situation not actually presented. 

The various forms of shock treatment — electric, insulin, 
metrazol —are less drastic methods of doing a very similar thing. 
They do not destroy brain tissue or at least are not intended 
to destroy it, but they do have a decidedly damaging effect on 
the memory. In so far as this concerns the circulating memory, 
and in so far as this memory is chiefly damaged for the recent 
period of mental disorder, and is probably scarcely worth pre- 
serving anyhow, shock treatment has something definite to re- 
commend it as against lobotomy ; but it is not always free from 
deleterious eftects on the permanent memory and the personality. 
As it stands at present, it is another violent, imperfectly er, 
stood, imperfectly controlled method to interrupl a menta 
vicious circle. This does not prevent its being in many cases 


the best thing we can do al present. " 

Pe ctgay aot shock trealment are methods which seer 
very nature are more suited to handle vicious ‘oe ‘a 6 
ignant worries than the deeper-seate ye 
ugh it is not impossible that they may have 
As we have said,in long-established cases 


memories and mal 
nent memories, tho 
some effect here too. 
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of mental disorder, the permanent memory is as badly deranged 
as the circulating memory. We do not seem to possess any 
purely pharmaceutical or surgical weapon for intervening 
differentially in the permanent memory. This is where psy- 
choanalysis and other similar psychotherapeutic measures 
come in. Whether psychoanalysis is taken in the orthodox 
Freudian sense or in the modified senses of Jung and of Adler, 
or whether our psychoterapy is not Strictly psychoanalytic 
at all, our treatment is clearly based on the concept that the 
stored information of the mind lies on many levels of acces- 
sibility, andis much richer and more varied than that which is 
accessible by direct unaided introspection ; that it is vitally 


conditioned by affective experiences which we cannot always 
uncover by such introspection, either b 


ecause they never were 
made explicit in our adult ] 


terpret these hidden memories, to make the patient accept them 
for what they are, and by their acceplance, modify, if not their 
content, at least the affective lone they carry, and thus make 
them less harmful. Al] this is perfectly consistent with the point 
of view of this book. It perhaps explains, too, why there are 
circumstances where a joint use of shock treatment and psycho- 
therapy is indicated, combining a physical or pharmacological 
therapy for the phenomena of reverberation in the nervous 
System, and a Psychological therapy for the long-time memories 
which, without interference, might re-establish from within the 
vicious circle broken up by the shock treatment. 
ie ge : he traffic Problem of the nervous 
: ed on by many wrilers, such as 
:that each form of Organization has an upper 
ill not function. ‘Thus the insect 
the length of tubing over which the 
air by diffusion directly to the brea- 


1. Thompson, D'Arcy, Growth and Form, Qua Ed. (1945) 
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thing tissues will function; a land animal cannot be so big that 
the legs or other portions in contact with the ground will be 
crushed by its weight; a tree is limiled by the mechanism for 
transferring water and minerals from the roots to the leaves, 
and the products of photosynthesis from the leaves to the roots; 
and so on. ‘The same sort of thing is observed in engineering 
constructions. Skyscrapers are limited in size by the fact that, 
when they exceed a certain height, the elevator space needed 
for the upper stories consumes an excessive part of the cross- 
section of the lower floors. Beyond a certain span, the best 
possible suspension bridge which can be built out of materials 
with given elastic properties will collapse under ils own weight ; 
and beyond a certain greater span, any structure built of a given 
material or materials will collapse under ils own weight. Simi- 
larly, the size of a single telephone central, built according to 
a constant, non-expanding plan, is limited, and this limitation 
has been very thoroughly studied by telephone engineers. 

In a telephone system, the important limiting factor is the frac- 
tion of the time during which a subscriber will find it impos- 
sible to puta call through. A 99 per cent chance of success will 
certainly be satisfactory for even the most exacting ; 90 per 
cent of successful calls is probably good enough to permit bus- 
iness to be carried on with reasonable facility. A success of 


75 per cent is already annoying, but will permit business to be 
carried on after a fashion ; while if half the calls end in failures, 
subscribers will begin to ask to have their telephones taken out. 
Now, these represent all-over figures. if the calls go through 
n distinct stages of switching, and probability of failure is inde- 


pendent and equal for each stage, in order to get a probability of 
total success equal to p, the probability of success al each mie 
must be p!/r. Thusto obtain a 75 per cent chance of the comple- 
tion of the call after five stages, We must have about 95 per cent 
chance of success per stage. To obtain 90 per cent of ae 
mance, we must have 98 per cent chance of performance at eac 
stage. To obtain 50 per cent of performance, we ua ma 
87 per cent of success at each stage. It will be a a 
more stages which are involved, the more rapidly the ser 


becomes extremely bad when a critical level of failure for i 
individual call is exceeded, and extremely good when this 
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critical level of failure is not quite reached. Thus a switching 
service involving many slages and designed fora certain level 
of failure shows no obvious signs of failure until the traffic 
comes up to the edge of the critical point, when it goes com- 
pletely to pieces, and we have a catastrophic traffic jam. 
Man, with the best developed nervous system of all the animals, 
with behavior that probably depends on the longest chains of 
effectively operated neuronic chains, is then likely to perform a 
complicated type of behavior efficiently very close to the edge 
of an overload, when he will give way in a serious and catas- 
trophic way. This overload may take place in several ways : 
either by an excess in the amount of traffic to be carried, by 4 
physical removal of channels for the carrying of traffic, or by 
the excessive occupation of such channels by undesirable systems 
of traffic, like circulating memories which have increased to the 
extent of becoming pathological worries. In all these cases, @ 
point will come — quite suddenly — when the normal traffic will 
not have space enough allotted to it, and we shall have a form 
of mental breakdown, very possibly amounting to insanity. 
This will first affect the faculties or operations involving the 
longest chains of neurons. There is appreciable evidence that 
these are precisely the Processes which are recognized to be the 
highest in our ordinary scale ofvaluation. The evidence is this: 
arise in temperature within nearly physiological limitsis known 
to produce an increase in the ease of performance of most if not 
of all neuronic processes. This is greater for the higher pro- 


cesses, roughly in the order of our usual estimate of their degree 
of « highness ». Now, any facilitation of 


neuron-synapse system should be cumula 
combined in series with other neurons, 
assistance a process receives through a rise in temperature is a 
rough measure of the length of the neuron chain it involves. 
We thus see that the Superiorily of the human brain to others 
in the length of the neuron chains it employs is a reason why 
mental disorders are certainly most Conspicuous and probably 
most common in man. There is another more specific way of 
considering a very similar matter. Let us first consider two 
brains geometrically similar, with the weights of gray and of 
white matter related by the same factor of proportionality, but 


a process in a single 
tive as the neuron is 
Thus the amount of 
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with different linear dimensions in the ratio A: B. Let the 
volume of the cell-bodies in the gray matter and the cross-sec- 
tions of the fibres in the white malter be ofthe same size in both 
brains. Then the number of cell-bodies in the two cases bears 
the ratio A®: B3, and the number of long-distance connectors the 
ratio A2:B2. This means that for the same density of activily 
in the cells, the density of activily in the fibres is A : B times as 
great in the case of the large brain as in that ofthe small brain. 

If we compare the human brain with that of a lower mammal, 
we shall find that il is much more convoluted. ‘The relative 
thickness of the gray matter is much the same, but it is spread 
over a far more involved system of gyri and sulci. The effect 
of this is to increase the amount of grey matter al the expense 
of the amount of white matter. Within a gyrus, this decrease 
of the while matter is largely a decrease in length rather than 
in number of fibres, as the opposing folds of a gyrus are nearer 
together than they would be ona smooth-surfaced brain of the 
same size. On the other hand, when it comes to the connectors 
between different gyri, the distance they have to run is increased 
if anything by the convolution of the brain. Thus the human 
brain would seem lo be fairly efficient in the matter of the short- 
distance connectors, but quite defective in the matter of long- 
distance trunk lines. This means that in the case of a traffic 
jam, the processes involving parts of the brain quite remote from | 
one another should suffer first. That is, processes involving 
several centers, 4 number of different motor processes, and a 
considerable number of association areas, should be among the 
least stable in cases of insanity. These are precisely the pro- 
cesses which we should normally class as higher, and we obtain 
another confirmation of our expectation, which seems to be 
verified by experience, that the higher processes deteriorate first 


in insanity. f 
There is some evidence thal the long-distance paths in the 
brain have a tendency to run outside of the cerebrum allogether, 
and to traverse the lower centers. This is indicated by the 
remarkably small damage done by culling some of the long- 
It almost seems as if 


distance cerebral loops of wile matter. 
these superficial connections were S° inadequate that they only 


furnish a small part of the connections really needed. 
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With reference to this, the phenomena of handedness and of 
hemispheric dominance are interesting. Handedness seems to 
occur in the lower mammals, though it is less conspicuous than 
in man, probably in part because of the lower degree of organi- : 
zation and skill demanded by the tasks which they perform. 
Nevertheless, the choice between the right and the left side in 
muscular skill does actually seem to be less than in man even 
in the lower primates. 

The right-handedness of the normal man, as is well known, is 
generally associated with a left-brainedness ; and the left-hand- 
edness of a minority of humans with a right-brainedness. That 
is, the cerebral functions are not distributed evenly over the 
two hemispheres, and one of these, the dominant hemisphere, 
has the lion’s share of the higher functions. [1 is true that many 
essentially bilateral functions — those involving the fields of 
vision, for example — are represented, each in its appropriate 
hemisphere, though this is not true for all bilater 


al functions. 
However, most of the « higher 


» areas are confined to the 
dominant hemisphere. For example, in the adult, the effect of 


an extensive injury in the secondary hemisphere is far less 
serious than the effect of a Similar injury in the dominant 
hemisphere. At a relatively early age in his career, Pasteur 
suffered a cerebral hemorrhage on his right side which left him 
with a moderate degree of one-sided paralysis, a hemiplegia. 
When he died, his brain was examined, and he was found 
to be suffering from a right-sided injury, so extensive that it 
has been said that after his injury, « he had only half a brain. » 
There certainly were extensive lesions of the parietal and 
temporal regions. Nevertheless, after this injury, he did some 
of his best work. A similar injury of the left side in a right- 
handed adult would almost certainly have been fatal, and would 


certainly reduce the patient into an animal condition of mental 
and nervous crippledness. 


It is said that the situation i 
infancy, and thal in the first si 
injury to the dominant hemisp 
secondary hemisphere to take it 
pears far more nearly normal t 
occurred at a later stage. 


S considerably better in early 
x months of life, an extensive 
here may compel the normally 
S place; so that the patient ap- 
han he would be, had the injury 
This is quite in accordance with the 
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general great flexibility shown by the nervous system in the 
early weeks of life, and the greal rigidity which it rapidly devel- 
ops later. It is possible that, short of such serious injuries, 
handedness is reasonably flexible in the very young child. How- 
ever, long before the child is of school age, the natural handed- 
ness and cerebral dominance are established for life. It used 
to be thought that left-handedness was a serious social disad- 
vantage. With most tools, school desks, and sports equipment 
primarily made for the right-handed, it certainly is to some 
extent. In the past, moreover, it was viewed with some of the 
superstitious disapproval that has attached to so many minor 
variations from the human norm, such as birthmarks or red 
hair. Fromacombination ofmotives, many people have atlemp- 
ted, and even succeeded, in changing the external handedness 
of their children by education, though of course they could not 
change its physiological basis in hemispheric dominance. It 
was then found that in very many cases these hemispheric 
changelings suffered from stuttering and other defects of speech, 
reading, and wriling, to the extent of seriously wounding their 
prospects in life and their hopes for a normal career. 

We now see at least one possible explanation for the phe- 
With the education of the secondary hand there 
ation of that part of the secondary 


hemisphere which deals with skilled motions, such as wri- 
tions are carried oul in the 


ting. Since however these mo 
closest possible association with reading, speech, and other acti- 
vities which are inseparably connected with the dominant hemis- 


phere, the neuron chains involved in processes of the sort must 
cross over from hemisphere to hemisphere and back ; and in a 
process of any complication, they must do this again and again. 
Now, the direct connectors between the hemispheres — the cere- 
bral commissures,---in a brain as large as that of man, are so 
few in number that they are of very little use; and the inter- 
hemispheric traffic must go by roundabout routes through the 
brain-stem ; which we know very imperfectly, but which are cer- 


scanly, and subject to interruption. As a conse- 
d with speech and writing are 


and stuttering is the 


nomenon. 
has been a partial educ 


tainly long, 
quence, the processes associate 
very likely to be involved in a traffic jam, 
most natural thing in the world. 
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That is, the human brain is probably too large already to use 
in an efficient manner all the facilities which seem to be anato- 
mically present. In a cat, the destruction of the dominant 
hemisphere seems to produce relatively less damage thanin man, 
and the destruction of the secondary hemisphere probably more 


damage. At any rate, the apportionment of function in the two 
hemispheres is more nearly equal. 


by the increase,in the size and co 
partly nullified by the fact that les: 
effectively at one time. 

he facing one of those | 
Specialized organs reacl 
ultimately lead to the 
brain may be as far alo 
lization as the great no 


In man, the gain achieved 
mplication of the brain, is 
Ss of the organ can be used 
It is interesting to reflect that we may 
imitations of nature, in which highly 
h a level of declining efficiency, and 
extinction of the species. The human 
ng on its road to this destructive specia- 
se horns of the last of the titanotheres. 


CHAPTER VIII 


INFORMATION, LANGUAGE, AND SOCIETY 


The concept of an organization, the elements of which are 
themselves small organizations, is neither unfamiliar nor new. 
The loose federations of ancient Greece, the Holy Roman empire 
and ils similarly constiluled feudal contemporaries, the Swiss 
Companions of the Oath, the United Netherlands, the United 
States of America and the many United States to the south of it, 
the Union of Socialist Soviet Republics, are all examples of 
hierarchies of organizations on the political sphere. The Levia-. 
than of Hobbes, the Man-State made up of lesser men, is an 
illustration of the same idea one stage lower in scale, while 


Leibniz’s is treatment of the living organism as being really a 
plenum, wherein other living organisms, such as the blood cor- 
her step in the same direction. 


puscles, have their life, is but anot 
It is in fact scarcely more than a philosophical anticipation of 


the cell theory, according to which most of the animals and 
plants of moderate size, and all of those of large dimensions, 
are made up of units, cells, which have many if not all the 


attributes of independent living organism. The multicellular 
organisms may themselves be the building-bricks of organisms 
of a higher stage, such as the Portuguese Man-of-War, which is 
complex structure of differentiated coelenterate polyps; where 


the several individuals are modified in different ways to serve 
the nutrition, the support, the excretion, the 
reproduction, and the support 0 


Strictly speaking, such a physicall ; ‘ 
poses no question of organization which is philosophically deep- 


er than those which arise at a lower level of individuality. ' It 
is very different with man and the olher social animals — with 
the herds of baboons oF cattle, the beaver colonies, the hives 


the locomotion, 
f the colony as 4 whole. 


y conjoint colony as that 
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of bees, the nests of wasps or ants. The degree of integration ot 
the life of the community may very well approach the level 
shown in the conduct of a single individual, yet the individual 
will probably have a fixed nervous system, with permanent 
topographic relations between the elements and permanent con- 
nections, while the community consists of individuals with 
shifting relations in space and time,and no permanent, unbreak- 
able physical connections. All the nervous tissue of the bee- 
hive isthe nervous tissue of some single bee : how then does the 
beehive act in unison, and at that in a very variable, adapted, 
organized unison ? Obviously the secret is in the intercommu- 
nication of its members, 

This intercommunication can vary greatly in complexity and 
content. With man, it embraces the whole intricacy of language 
and literature, and very much besides. With the ants, it pro- 
bably does not cover much more than a few smells. It is very 
improbable that an ant can distinguish one ant from another. 

_It certainly can distinguish an ant from its own nest from an 
ant froma foreign nest, and may cooperate with the one, destroy 
the other. Within a few oulside reactions of this kind, the ant 
Seems to have a mind almost as patterned, chitin-bound, as its 
body. ltis what we might expect a Priori from an animal whose 
growing phase, and to a large extent, whose learning phase, is 
rigidly separated from the phase of mature activity. The only 


means of communication we can trace in them 


are as general 
and diffuse as the h 


Ormonal system of communication within 
smell, one of the chemical 


senses, general 
S itis, is not unlike the horm 


onal influences 


s, indis- 
bringing the 
time, and serve for the conti- 
By this I do not mean to assert that the 
substances, once they reach the organ of 
ther than nervous. It is hard to see how 


hormonal in quantities as small as those which 
are readily perceivable ; on the other hand, we know too little 


inner action of these 
smell, ishormonal ra 
it can be purely 


} 
) 
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of the action of the hormones to deny the possibility of the 
hormonal action of vanishingly small quantities. of such sub- 
stances. Moreover, the jong, twisted rings of carbon atoms 
found in muskone and civetone do not need too much rearrange- 
ment to form the linked ring structure characteristic of the 
sex hormones, some of the vitamins, and some of the carcino- 
gens. I do not care to pronounce an opinion on this matter ; 
I leave it as an interesting speculation. 

The odors perceived by the ant seem to lead to a highly 
standardized course of conduct ; but the value of a simple 
stimulus such as an odor for convering information depends, 
not only on the information conveyed by the stimulus itself, but 
on the whole nervous conslitution of the sender and the receiver 
of the stimulus as well. Suppose | find myself in the woods 
with an intelligent savage, who cannot speak my language, and 
whose language I cannot speak. Even without any code of 
sign language common to the two of us, | can learn a greal deal 


from him. All1 need to dois to be alert to those moments when 


he shows the signs of emolion or interest. I then cast my eyes 
around, perhaps paying special atiéntion (o the direction of 
his glance, and fix in my memory what I see or hear. It will 
not be long before I discover the things which seem important 
to him, not because he has communicated them to me by lan- 
guage, but because I myself have observedthem. Inother words, 
a signal wilhout an intrinsic conient may acquire meaning in 
his mind by what he observes at the time, and may aquire mea- 
ninginmy mind by what] observe at the time. The ability thathe 
has to pick out the moments of my special, active attention is in 
itself a language as varied in possibilities as the range of impres- 


sions that the two of us are able to encompass. Thus social 


animals may have an active, intelligent, flexible means of com- 


munication long before the development of language. 
Whatever means of communication the race may have, it is 
and to measure the amount of information 
and to distinguish it from the amount of 
Certainly no informa- 


possible to define 
available to the race, 
information available lo the individual. 
tion available to the individual is also available to the race, 


unless it modifies the behavior of one individual to another, nor 
is even that behavior of racial significance, unless it is distin- 
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guishable by other individuals from other forms of behavior. 
Thus the question as to whether a certain piece of information is 
racial, or of purely private availability, depends on whether it 
results in the individual assuming a form of activily which can 
be recognized as a distinct form of activity by other members 
of the race, in the sense that it will in turn affect their activity, 
and so on. 

I have spoken of the race. This is really too broad a term 
for the scope of most communal information. Properly spea- 
king, the community extends only so far as there extends an 


effectual transmission of information. It is possible to give a 


sort of measure to this, by comparing the number of decisions 
entering a group from outside with the number of decisions made 
in the group. We can thus measure the autonomy of the group. 
A measure of the effective size of a group is given by the size 
which it must have to have achieved a certain stated degree of 
autonomy. 

A group may have more group information or less group 
information than its members. A group of non-social animals, 
temporarily assembled, contains very little group information, 


d on’ by them in a way that 
other hand, the human orga- 
ation, in all probability, than 
There is thus no necessary relation 
n the amount of racial or tribal or 


community information and the-amount of information available 


to the individual. 


As in the case of the individual, not all the information which 


is available to the race at one time is accessible without special 
effort. There is a well-known tendency of libraries to become 
clogged by their own velume ; of the sciences to develop such 
a degree of specialization that the expert is often illiterate out- 
side his own minute specialty. Dr, Vannevar Bush has sugges- 
led the use of mechanical aids for the searching through vast 
bodies of material. These probably have their uses, but they 
are limited by the impossibility of classifying a book under an 
unfamiliar heading unless some particular person has already 


INFORMATION, LANGUAGE, AND SOCIETY 4185 


recognized the relevance of that heading for that particular book. 
In the case where two subjects have the same techniques and 
intellectual content, but belong to widely separated fields, this 
still requires some individual with an almost Leibnizian catho- 
licity of interest. 

In connection with the effective amount of communal infor- 
mation, one of the most surprising facts about the body politic 
is its extreme lack of efficient homeostatic processes. There is 
a belief, current in many countries, which has been elevated 
to the rank of an official article of faith in the United States, 
that free competition is itself a homeostatic process : thal ina 
free market, the individual selfishness of the bargainers, each 
seeking to sell as high and buy as low as possible, will result 
in the end in a stable dynamics of prices, and with redound 
to the greatest common good. This is associated with the very 
comforting view that the individual entrepreneur, in seeking 
to forward his own interest, is in some manner a public bene- 
factor, and has thus earned the great rewards with which so- 
ciety has showered him. Unfortunately, the evidence, such as 
it is, is against this simple-minded theory. The market is a 
game, which has indeed received a simulacrum in the family 
game of Monopoly. It is thus strictly subject to the general the- 
ory of games, developed by von Neumann and Morgenstern. 
This theory is based on the assumption that each player, at 
in view of the information then available to him, 
a completely intelligent policy, which 
will in the end assure him of the greatest possible expectation 
of reward. It is thus the market game as played between perfec- 
tly intelligent, perfectly ruthless operators. Even in the case 
of two players, the theory is complicated, although it often leads 
to the choice of a definite line of play. In many cases, however, 
where there are three players, and in the overwhelming majo- 
rity of cases, when the number of players is large, the Tesult 
is one of extreme indeterminacy and instability. The indivadia] 
players are compelled by their own cupidily to form nee 
but these coalitions do not generally establish themse eM 
any single, determinate Way, and usually ica tthe: ew 
of betrayal, turncoatism, and decepuon; which is ae 4 Ht As © 
a picture of the higher business life, or the closely related lives 


every slage, 
playsin accordance with 
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of politics, diplomacy, and war. In the long run, eren the most 
brilliant and unprincipled huckster must expect ruin ; but let 
the hucksters become tired of this, and agree to live in peace 
with one another, and the greal rewards are reserved for the 


one who watches for an opportune time to break his 


agreement 
and betray his companions. 


There is no homeostasis whatever. 
We are involved inthe business cycles of boom and failure, in 


the successions ofdictalorship and revolution, in the wars which 
everyone loses, which are so real a feature of modern times. 
Naturally, von Neumann’s picture of the player as a completely 
intelligent, completely ruthless person, is an abstraction and 2 
perversion of the facts. It is rare*lo find a large number of 
thoroughly clever and unprincipled persons playing a game 
together. Where the knaves assemble, there will always be 
fools; and where the fools are present in sufficient numbers, 
they offer a more profitable object of exploitation for the knaves. 
The psychology of the fool has become a subject well worth the 
serious altention of the knaves. Instead of looking out for his 
own ultimate interest, after the fashion 
Sters, the fool Operates in a m 
predictable as the Struggles of a rat in a maze. 
lies — or rather, of statements irreley 
him buy a particular brand of cig 
the party hopes, induce him to 
any candidate — or tojoin ina 
precise mixture of religion, 
will sell an illustrated newspa 
bribery, and intimidation wil 
on guided missiles or the 


of von Neumann’s game- 
anner which, by and large, is as 
This policy of 
ant to the truth — will make 
aretles; thal policy will, or so 
vote for a particular candidate 
political witch hunt. A certain 
pornography, and pseudo-science 
per. Acertain blend of wheedling, 
| induce a young scienlist to work 


atomic bomb. To determine these, we 
have our machinery of radio fan-ratings, straw votes, opinion 


samplings, and other Psychological investigations with the com- 
mon man as their object ; and there are alwa 


sociologists, and economists available to se 
these undertakings, 


Luckily for us, these merch 


gullibility, have not yet arrived at such a pitch of perfection as 
to have things all their own way. This is bec 
either all fool or all knave. The average man is q 


cts which come toh 


ause no man is 
uile reasonably 
is direct atten- 
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tion, and quite reasonably altruistic in matters of public benefit 
or private suffering which are brought before his owa eyes. Ina 
small country communily which has been running long enough 
lo have developed somewhat uniform levels of intelligence and 
behavior, there is a very respectable standard of care for the 
unfortunate, of administration of roads and other public facilities, 
of tolerance for those who have offended once or twice against 
society. After all, these people are there, and the rest of the 
community must continue to live with them. On the other 
hand, in such a community, it does not do for a man to have 
the habit of over-reaching his neighbors. There are ways of 
making him feel the weight of public opinion. After a while, he 
will find it so ubiquitous, so unavoidable, so restricting and 
oppressing, that he will have to leave the community in self 
defense. 

Thus small, closely knit communities have a very considerable 
measure of homeostasis ; and this, whether they are highly literate 
communities in a civilized country, or villages of primitive 
savages. Strange and even repugnant as the customs of many 


eem to us, they generally have a very definite 


barbarians may s 
anthropolo- 


homeostatic value, which it is part of the function of 
It is only in the large community, where the 
Ives from hunger by 
anonymity, from 
ession of the 


gists to interpret. 
Lords of Things as They Are protect themse 
wealth, from public opinion by privacy and 
private crilicism by the laws of libel and the poss 
means of communication, that ruthlessness can reach its most 
sublime levels. Ofall of these anti-homeostatic factors in society, 
the control of the means of communication is the most effective 


and most important. 
One of the lessons of the present book is that any organism 


is held together in this action by the possession of means for 
the acquisition, use, retention, and transmission of information. 
In a society too large for the direct contact of ils members, these 
means are the press, both as itconcerns books and asit concerns 
the radio, the telephone system, the telegraph, the 


newspapers, 
the schools, and the church. 


posts, the theater, the movies, 
Besides their intrinsic importance as means of communication, 


es other, secondary functions. The newspaper 


each of these serv 
dan instrument for the monetary 


is a vehicle for advertisement, an 
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gain of its proprietor, as are also the movies and the radio. ‘The 
school and the church are not merely refuges for the scholar and 
the saint: they are also the home of the Great Educator and the 
Bishop. The book that does not earn money for its publisher 
probably does not get printed, and certainly does not get 
reprinted. ; 

In a society like ours, avowedly based on buying and selling, 
in which all natural and human resources are regarded as the 
absolute property of the first business man enterprising enough 
to exploit them, these Secondary aspects of the means of com- 
munication tend to encroach further and further on the primary 
ones. This is aided by the very elaboration and the consequent 
expense of the means themselves. The country paper may 
continue to use its own reporters to canvass the villages around 
for gossip, but it buys its national news, its syndicated features, 
its political opinions, as stereotyped « boiler plate». The radio 
depends on its advertisers for income, and as everywhere, the 
man who pays the piper calls the tune. The great news services 
cost too much to be available to the publisher of moderate means. 
The book publishers concentrate on books that are likely to be 
acceptable to some book club which buys out the whole of an 


enormous edition. The college president and the Bishop, even 
if they have no personal ambitions for power, have expensive 
inslitulions to run, and ¢ 


an only seek their money where the 
money is. 


Thus on all sides we have a triple constriction of the means of 


communication : the elimination of the less profitable means in 
favor of the more profitable ; the fact that these means are in the 
hands of the very limited el 


ass of wealthy men, and thus natu- 
vally express the Opinions of that class +and the further fact that, 
as one of the chief avenues to political and personal power, they 
attract above all those ambitious for such power. ‘Chat system 
which more than all others should contribute to soci 
lasis is thrown directly into 
in the game of power and money, 
be one of the chief anti-homeostat 
It is no wonder, then, that the 1 


ee 
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human elements of which all communities are built up. Like 
the wolf-pack, although let us hope to a lesser extent, the State 
is stupider than most of its components. 

This runs counter to a tendency much voiced among 
business executives, heads of great laboratories, and the like, 
to assume that because the community is larger than the 
individual, it is also more intelligent. Some of this opinion is 
due to no more than a childish delight in the large and the 
lavish. Some of itis due to a sense of the possibilities of a large 
good. Not a little of it, however, is nothing 


organization for 
the main chance and a lusting after the 


more than an eye for 
fleshpots of Egypt. 

There is another 
anarchy of modern society 
that there must be some way out has led to an overv 
the possible homeostatic elements in the community. Much as 
we may sympathize with these individuals, and appreciate the 
emotional dilemma is which they find themselves, we cannot 
attribute too much value to this type of wishful thinking. Itis the 
mode of thought of the mice when faced with the problem of 
belling the cat. Undoubtedly it would be very pleasant for us 
mice ifthe predatory cats of this world were to be belled, but — 
who is going to do it? Who is to assure us that ruthless power 
will not find its way backinto the hands of those most avid forit? 

I mention this matter, because of the considerable, andI think 
false, hopes which some of my friends have built for the social 
efficacy of whatever new ways of thinking this book may 
They are certain that our control over our material 
outgrown our control over our social 
anding thereof. Therefore, they 
rediate future is to extend 


group of those who see nothing good in the 
, and in whom an optimistic feeling 
aluation of 


contain. 
environment has far 
environment and our underst 
consider that the main task of the imm 
to the fields of anthropology, of sociology, of economics, the 
methods of the natural sciences, in the hope of achieving a like 
measure of success in the social fields. From believing this 
necessary, they come to believe it possible. hy this, I ee 
they show an excess and a misunderstanding © 


the nature of all scientific ach 
All the great successes in precise s 
fields where there is a certain high 


ive optimism, 
ievemenlt. 
cience have been made in 


degree of isolation of the 
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phenomenon from the observer. We have seen in the case of 
astronomy that this may result from the enormous scale of 
certain phenomena with respect to man, so thal man’s mightiest 
efforts, not to speak of his mere glance, cannot make the slightest 
visible impression on the celestial world. In modern atomic 
physics, on the other hand, the science of the unspeakably 
minute, it is true that anything we do will have an influence 
on many individual particles which is great from the point of view 
of that particle. However, we do not live on the scale of the 
particles concerned, either in space or in time ; and the events 
that might be of the greatest signific 
of an observer conforming to their scale of existence, appear to 
us — with some exceptions, it is true, as in the Wilson cloud- 
chamber experiments, only as average mass effects in which 
enormous populations of particles cooperate. As far as these 
effects are concerned, the intervals of time concerned are large 
from the point of view of the individual particle and its motion, 
and ourstalistical theories have an admirably adequate basis. In 
short, we are too small to influence the stars in their courses, and 
too large to care about anything but the mass effects of molecules, 
atoms, and electrons. In both cases, we achieve 
loose coupling with the phenomena we are 
Massive total account of this co 


ance from the point of view 


a sufficiently 
studying to give a 
upling, although the coupling may 
be able to ignore it altogether. 


er is able to exert a considerable 
that come to his attention, With 
€, skill, and honesty of purpose of 


> * cannot think that any community 
which they have investigated will ever be quite the same 


afterwards, Many a missionary has fixed his own misunder- 
Standings of a primitive language as law eternal in the process 
of reducing it to writing. There is much in the social habits of 
a people which is dispersed and distorted by the mere act of 


making inquiries about it. In another sense from that in which 
it is usually stated, tradutiore traditore. 
On the other hand, the soci 


al scientist has not the advantage 
of looking down on his subjec 


ts from the cold heights of elernily 
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and ubiquity. It may be that there is a mass sociology of the 
human animalcule, observed like the populations of Drosophila 
in a bottle, but this is not a sociology in which we, who are 
human animalcules ourselves, are particularly interested. We 
are not much concerned about human rises and falls, pleasures 
and agonies, sub specie aeternitalis. Your anthropologist reports 
the customs associated with the life, education, career, and death 
of people whose life-scale is much the same as his own. Your 
economist is most interested in predicting such business cycles 
as run their course in less than a generation ; or at least, have 
repercussions which affect a man differentially at different stages 
of his career, Few philosophers of politics nowadays care to 
confine their investigations to the world of Ideas of Plato. 

In other words, in the social sciences we have to deal with 
short statistical runs, nor can we be sure that a considerable 
part of what we observe is not an artefact of our own creation. 
An investigation of the stock market is likely to upset the stock 
We are too much in tune with the objects of our 
e good probes. In short, whether our investi- 
al sciences be statistical or dynamic —and they 
should participate in the nature of both — they can never be good 
to more than a very few decimal places, and, in short, can never 
furnish us with a quantily of verificable, significant information 
which begins to compare wilh that which we have learned to 
expect in the natural sciences. We cannot afford to neglect 


them ; neither should we build exaggerated expectations of their 
There is much which we must leave, whether we 
e method of the 


market. 
investigation to b 
gations in the soci 


possibilities. 
like it or not, to the un- « scientific », narrativ 


professional historian. 


NOTE 


There is one question which properly belongs to this chapter, 
though it in no sense represents a culmination of its argument. 
It is the question whether it is possible to construct a chess- 
playing machine, and whether this sort of ability represents an 
essential difference between the potentialities of the machine and 
the mind. Note that we need not raise the question as to whether 
it is possible to construct a machine which will play an optimum 
game in the sense of von Neumann. Not even the best human 
brain approximates to this. At the other end,it is unquestion- 
ably possible to construct a machine that will play chess in 
the sense of following the rules of the game, irrespective of the 
merit of the play. This is essentially no more difficult than the 
truction of a system of inlerlocking signals for a railway 


cons 
signal tower. The real problem is intermediate : lo construct 


all offer inleresting opposition to a player 


a machine which sh 
any levels at which human chess players 


at some one of the m 
find themselves. 

[ think it is possible 
rivial, apparatus for this purpos 
play — ata high speed if possi 
e opponent's admissible ripostes 
for two or three moves ahead. To each sequence of moves it 
should assign a certain conventional valuation. Here, to 
checkmate the opponent receives the highest valuation at each 
stage, to be checkmated the lowest ; while losing pieces, taking 
checking, and other recognizable situations, 
t too remote from those which 
The first ofan entire sequence 
Yeumann’s 


to construct a relatively crude, but not 


altogether t e. The machine 
must actually ble — all its own 


admissible moves and all th 


opponents’ pieces, 
should receive valuations no 
good players would assign them. 
of moves should receive a valuation much as von? 
theory would assign it. At the stage at which the machine is 
to play once and the opponent once, the valuation of a play by 
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the machine is the minimum valuation of the situation after the 
Opponent has made all possible plays. At the stage where ihe 
machine is to play twice and the Opponent twice, the valuation 
ofa play by the machine is the minimum witl 
opponent's first play of the maximum valu 
the machine at the stage when there is onl 
ponentand one by the machine to follow. 
extended to the case when each player m 
soon. Then the machine chooses any 
the maximum valuation for the st 
has some value on which the designer of the machine has de- 
cided. This it makes as its definitive play. 

Such a machine would not only play legal chess, but a chess 
not so manifestly bad asto be ridiculous. At any slage, if there 
were a mate possible in two or three moves, the machine would 
make it ; and if it were possible to avoid a mate by the enemy in 
two or three moves, the machine would avoid it. It would pro- 
bably win over a stupid or careless chess player, and would almost 
cerlainly lose to a careful player of any considerable degree of 
proficiency, Inother words, it mightvery well beas good aplayer 
as the vast majority of the human race. This does not mean 
that it would reach the degree of proficiency of Maelzel’s frau- 


dulent machine, but for all that, it may attain a pretty fair 
level of accomplishment. 


h respect to the 
ation of the plays by 
y one play of ithe op- 

This process can be 
akes three plays, and 
one of the plays giving 
age n plays ahead, where n 
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